department of management … design (lecture 3) mar21 a… · department of management...

122
調查研究 調查研究 K th S L (罗胜强) Kenneth S . Law (罗胜强) Department of Management The Chinese University of Hong Kong The Chinese University of Hong Kong 香港中文大学管理系 [email protected] http://kennethlaw.blog.chinahrd.net/

Upload: vannga

Post on 19-May-2018

216 views

Category:

Documents


1 download

TRANSCRIPT

調查研究調查研究

K th S L (罗胜强)Kenneth S Law (罗胜强)

Department of ManagementThe Chinese University of Hong KongThe Chinese University of Hong Kong

香港中文大学管理系mnlawcuhkeduhk

httpkennethlawblogchinahrdnet

Common Methods in Mgt Research2

Common Methods in Mgt Research1 Qualitative reviews2 Qualitative studies

Case studiesContent analysisContent analysis

3 Experimental laboratory studiesANOVA regression

4 Quasi experiments5 Correlational survey studies

i SEM HLMregression SEM HLM6 Meta analysis7 Field study Content analysis

Kenneth Law 同济大学 2010

7 Field study Content analysis Database helliphellip

1 Qualitative Reviews3

1 Qualitative Reviews

C R iContent ReviewTheory buildingHypotheses developmentHypotheses development

Manor B (2002) International assignmentsManor B (2002) International assignments for career building A model of agency relationships and psychological contracts Academy of Management Review 27(3) 373Academy of Management Review 27(3) 373-391

Kenneth Law 同济大学 2010

Abstract4

Abstract

We present a theoretical model of international assignmentsin which we examine the alignment or nonalignment of the organizationrsquos and individualrsquos expectations of an assignmentorganization s and individual s expectations of an assignment and its effect on assignment success We consider four basic configurations in our model ndash mutual loyalty mutual transaction agent opportunism and principal opportunism ndashtransaction agent opportunism and principal opportunismand present a matrix of organization-individual alignment to help predict varying degrees of success in expatriate assignment and in repatriation Finally we apply this matrixassignment and in repatriation Finally we apply this matrix as a framework for understanding changes in the employment contract over time

Kenneth Law 同济大学 2010

The Model (Agency Theory)5

Relational Transactional

Individual

Cell I Mutual loyalty Cell II Agent opportunism

High organizational success in expatriation and repatriation (P1)Hi h i di id l i t i ti

Moderate organizational success in expatriation but failure in repatriation (P3)i h i di id l i i iel

ationa

l

ganiza

tion

Cell III Principal opportunism Cell IV Mutual transaction

High individual success in expatriation and repatriation (P2)

High individual success in expatriation but mixed success in repatriation (P4)

Re

Org

Moderate organizational success in expatriation and low success in repatriation (P5)Moderate individual success in

Moderate to high organizational success in expatriation and a better chance of organizational success in repatriation than that in the case of misalignment (P7)Moderate to high individual success in ns

action

al

expatriation but failure in repatriation (P6)

Moderate to high individual success in expatriation and a better chance of individual success in repatriation than that in the case of misalignment (P8)

Tra

Kenneth Law 同济大学 2010

2 Qualitative Studies6

2 Qualitative Studies

S tt R I (1991) M i t i i b t dSutton RI (1991) Maintaining norms about expressed emotions The case of bill collectors Administrative Science Quarterly 36(2) 245-269

Kenneth Law 同济大学 2010

7

AbstractA qualitative study of a bill-collection organization is used to identify norms about the emotions that collectors are expected to convey to debtors and theabout the emotions that collectors are expected to convey to debtors and the means used by the organization to maintain such norms given that collectors expressed emotions are simultaneously influenced by their inner feelings The data indicate that collectors are selected socialized and rewarded for following the general norm of conveying urgency (high arousal with a hint offollowing the general norm of conveying urgency (high arousal with a hint of irritation) to debtors Collectors are further socialized and rewarded to adjust their expressed emotions in response to variations in debtor demeanor These contingent norms sometimes clash with collectors feelings toward debtors Bill collectors are taught to cope with such emotive dissonance by using cognitivecollectors are taught to cope with such emotive dissonance by using cognitive appraisals that help them become emotionally detached from debtors and by releasing unpleasant feelings without communicating these emotions to debtors

Kenneth Law 同济大学 2010

3 Laboratory designs8

3 Laboratory designs

Manipulation variables (independent variables)Manipulation variables (independent variables)Outcome variables (dependent variables)

Research questionHow would a set of outcome variables change as a result of our manipulation of some causalas a result of our manipulation of some causal variables (causality) lightning and thundering

Leadership style Employee performance

Kenneth Law 同济大学 2010

Examples9

Examples

Allen TD amp Rush MC (1998) The effects of organizational citizenship behavior on performance j d t A fi ld t d d l b t i tjudgments A field study and a laboratory experiment Journal of Applied Psychology 83(2) 247-260

Kenneth Law 同济大学 2010

10

AbAbstract

The process of linking organizational citizenship behavior (OCB) with performance judgments was investigated in a field and a laboratory study In the field study managers rated the task performance and OCB of 148 subordinates In the laboratory research 136 students viewed and ratedsubordinates In the laboratory research 136 students viewed and rated videotaped segments of teaching performance that demonstrated either high or low task performance and high or low OCB

Citizenship Behaviors Overall evaluationBehaviors

Kenneth Law 同济大学 2010

11

High OCB Low OCB

Manipulation

Results

Performance rating Performance ratingBy students (X1) gtgtgt By students (X2)

Kenneth Law 同济大学 2010

Types of experimental designs12

Types of experimental designs

1 Experimental designs

2Quasi experimentsbull Field experiments bull Time series designs

Kenneth Law 同济大学 2010

Experimental designs13

Experimental designsPretestposttest control-group designeg having one group of employees taking the training program in week 1 The second group which will get the training after group one is used as the control group

Why do we need a control group if we have pretest and posttest alreadyWhats wrong with just having posttest with control groups

XT1 T2Experimental group 1 2Experimental group

Control group T1 T2

Kenneth Law 同济大学 2010

Quasi experiment14

Quasi-experiment

- Experimental design without real manipulations- No randomization

- Hui C Lam SK amp Law KS (2000) Instrumental values of organizational citizenship behavior for promotion A field quasi-experiment Journal of Applied Psychology 85 822-828

Kenneth Law 同济大学 2010

15

Employees

P ti OCB High OCB Promotion OCB

Antecedents

Low OCB No Promotion OCB

Kenneth Law 同济大学 2010

Correlational surveys16

Correlational surveys

Predictor variables (independent variables)Criterion variables (dependent variables)

Research questionHow would a set of predictor variables affect a set of outcome

variables (causality)

Performance Promotion

time 1 time 1

Kenneth Law 同济大学 2010

Examples17

Barling J LughlinC amp Kelloway EK (2002) Development and test of a model linking safety-specific transformational leadership and occupational safety Journal of Applied Psychology 87(3) 488-496

Vancouver JB Thompson CM Tischner EC amp Putka DJ (2002) Two studies examining the negative effect of self-efficacy on performance Journal of Applied Psychology y p J f App y gy87(3) 506-516

Dvir T Eden D Avoilio BJ Shamir B (2002) ImpactDvir T Eden D Avoilio BJ Shamir B (2002) Impact of transformational leadership on follower development and performance A field experiment Academy of Management Journal 45(4) 735-744

Kenneth Law 同济大学 2010

Journal 45(4) 735-744

Survey study18

Survey study

rSafety‐specific

Occupational

rxy

transformational leadership

Occupational safety

Self‐efficacy Job performance

Transformational leadership

Job performance

Kenneth Law 同济大学 2010

Meta Analysis19

Meta Analysis

R h tiResearch questionCan we combine quantitative research in the past and summarize the research findings quantitatively

Judge T Heller D Mount M (2002) Five-Factor g ( )model of personality and job satisfaction A meta-analysis Journal of Applied Psychology 87(3) 530-541

Kenneth Law 同济大学 2010

Meta analysis20

y

Big‐5 personality individualism neuroticism conscientiousness agreeableness openness to experiences

Rxy is the correlation between agreeableness and job satisfaction

Study N rxy

1 150 35 2 247 ‐07

3 386 174 124 485 278 025 76 450 29

Kenneth Law 同济大学 2010

21

Kenneth Law 同济大学 2010

Issues in survey design 22

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Issues in survey design 23

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

1 What is your research question24

1 What is your research question Is the research question testableq

改善我国旅游业未来发展的几个意见

Are the constructs well defined 「企业进取性」和企业业绩表现的关系

Do we have validated scales to measure the constructs Existing scales Western scales

Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础

Kenneth Law 同济大学 2010

What is the research question25

qIs EQ really useful in predicting work outcomes

Would a firm using a strategic human resource management approach be more competitive

What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC

Can supervisors distinguish task performance from contextual performance in the PRC

What are the antecedents of contextual performance

Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC

Kenneth Law 同济大学 2010

生涯管理要素圖26

生涯情況 信息 計劃 資源目標

探索的動機 候迭人的信息

生活的目標

1個人戰略2時間

1 解決個人問題的技能

2 控制1自我評估價值技術興趣經驗

2組織評估表現潛力分配計劃

生涯發展目標 生涯計劃執行

分配計劃

現有內部勞資市場1工作調整需要2生涯之路結構3內部提升

生涯機會信息1生涯信息系統2生涯咨詢

1未來組織經濟目標

2未來所需職工

1組織人才資源發展戰略

2重要職務分配

1工作機會2贊助人3生涯管理者

Kenneth Law 同济大学 2010

Th di ti l f LMX

27

The mediating role of LMX

OCB

TransformationalLeadership LMX-MDM

OCB

77

3280

TaskPerformance

16

Kenneth Law 同济大学 2010

Contributions28

Contributions1 Theoretical contributions

bull New variables based on theorybull New perspectivesbull New findings

h d l l b2 Methodological contributionsbull New measures (eg new scale development)

N th d ( l l h)bull New methods (eg cross‐level research)

Kenneth Law 同济大学 2010

Identifying Research Questions29

Identifying Research Questions

员 企业的交换契约

公平理论与员工公民行为

Literature

员工企业的交换契约Incremental theoretical

contribution

Research Questions

M h d l l Practitioners公司治理

Methodological rigor

59岁现象

合资企业员工本土化

Kenneth Law 同济大学 2010

What is a research question30

What is a research question

My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust

How do you like that

Expected returnknowledge sharing

knowledge sharing

Benevolence

Expected return

Trust

s a gtendency

s a gbehaviors

Kenneth Law 同济大学 2010

A conversation hellip31

A conversation hellip

Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate

ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic

ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal

Kenneth Law 同济大学 2010

A different view32

A different view

We do research because we want to know We do research because we want to know the answer hellip

Therefore before we start a study we must have a research question

We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question

Kenneth Law 同济大学 2010

A different view33

A different view

With a research question you would automatically ask for a theoryautomatically ask for a theory

B d h ld Based on your theory you would automatically develop some hypotheses

Kenneth Law 同济大学 2010

The research process34

The research process

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Trusttendency behaviors

Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g

Kenneth Law 同济大学 2010

The research process35

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

H d k th t k l d h i i t ti i

Trust

How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g

This is because helliphellip (a theory)

Kenneth Law 同济大学 2010

The research process36

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Y d th k l d h i i t ti b

Trust

You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)

Kenneth Law 同济大学 2010

The research process37

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

(1) Some people have an in‐born tendency to share with

Trust

(1) Some people have an in‐born tendency to share with others

(2) Some people would expect reciprocal sharing in the future

(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)

(4) helliphellip

Kenneth Law 同济大学 2010

(4) helliphellip

Some questions38

1 Has anyone done anything on this topic before (literature review)

2 Which one is the most important reason to explain knowledge sharing (theory)

3 Are there established management theories that can explain the phenomenon (theory)

4 Are these antecedents or causes (theory)

5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)

6 What is the most appropriate research design to answer my research questionresearch question

Kenneth Law 同济大学 2010

When to use which design39

When to use which design

1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis

Kenneth Law 同济大学 2010

Do Not start your research40

Do Not start your research helliphellip

With some hypothesized relationships

With a model

Just because no one has done that before

Only because you see a significant correlation in your datasetyour dataset

Without looking at the literature

Without a research questionq

If you do not find it interesting

If you do not believe it is important

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Common Methods in Mgt Research2

Common Methods in Mgt Research1 Qualitative reviews2 Qualitative studies

Case studiesContent analysisContent analysis

3 Experimental laboratory studiesANOVA regression

4 Quasi experiments5 Correlational survey studies

i SEM HLMregression SEM HLM6 Meta analysis7 Field study Content analysis

Kenneth Law 同济大学 2010

7 Field study Content analysis Database helliphellip

1 Qualitative Reviews3

1 Qualitative Reviews

C R iContent ReviewTheory buildingHypotheses developmentHypotheses development

Manor B (2002) International assignmentsManor B (2002) International assignments for career building A model of agency relationships and psychological contracts Academy of Management Review 27(3) 373Academy of Management Review 27(3) 373-391

Kenneth Law 同济大学 2010

Abstract4

Abstract

We present a theoretical model of international assignmentsin which we examine the alignment or nonalignment of the organizationrsquos and individualrsquos expectations of an assignmentorganization s and individual s expectations of an assignment and its effect on assignment success We consider four basic configurations in our model ndash mutual loyalty mutual transaction agent opportunism and principal opportunism ndashtransaction agent opportunism and principal opportunismand present a matrix of organization-individual alignment to help predict varying degrees of success in expatriate assignment and in repatriation Finally we apply this matrixassignment and in repatriation Finally we apply this matrix as a framework for understanding changes in the employment contract over time

Kenneth Law 同济大学 2010

The Model (Agency Theory)5

Relational Transactional

Individual

Cell I Mutual loyalty Cell II Agent opportunism

High organizational success in expatriation and repatriation (P1)Hi h i di id l i t i ti

Moderate organizational success in expatriation but failure in repatriation (P3)i h i di id l i i iel

ationa

l

ganiza

tion

Cell III Principal opportunism Cell IV Mutual transaction

High individual success in expatriation and repatriation (P2)

High individual success in expatriation but mixed success in repatriation (P4)

Re

Org

Moderate organizational success in expatriation and low success in repatriation (P5)Moderate individual success in

Moderate to high organizational success in expatriation and a better chance of organizational success in repatriation than that in the case of misalignment (P7)Moderate to high individual success in ns

action

al

expatriation but failure in repatriation (P6)

Moderate to high individual success in expatriation and a better chance of individual success in repatriation than that in the case of misalignment (P8)

Tra

Kenneth Law 同济大学 2010

2 Qualitative Studies6

2 Qualitative Studies

S tt R I (1991) M i t i i b t dSutton RI (1991) Maintaining norms about expressed emotions The case of bill collectors Administrative Science Quarterly 36(2) 245-269

Kenneth Law 同济大学 2010

7

AbstractA qualitative study of a bill-collection organization is used to identify norms about the emotions that collectors are expected to convey to debtors and theabout the emotions that collectors are expected to convey to debtors and the means used by the organization to maintain such norms given that collectors expressed emotions are simultaneously influenced by their inner feelings The data indicate that collectors are selected socialized and rewarded for following the general norm of conveying urgency (high arousal with a hint offollowing the general norm of conveying urgency (high arousal with a hint of irritation) to debtors Collectors are further socialized and rewarded to adjust their expressed emotions in response to variations in debtor demeanor These contingent norms sometimes clash with collectors feelings toward debtors Bill collectors are taught to cope with such emotive dissonance by using cognitivecollectors are taught to cope with such emotive dissonance by using cognitive appraisals that help them become emotionally detached from debtors and by releasing unpleasant feelings without communicating these emotions to debtors

Kenneth Law 同济大学 2010

3 Laboratory designs8

3 Laboratory designs

Manipulation variables (independent variables)Manipulation variables (independent variables)Outcome variables (dependent variables)

Research questionHow would a set of outcome variables change as a result of our manipulation of some causalas a result of our manipulation of some causal variables (causality) lightning and thundering

Leadership style Employee performance

Kenneth Law 同济大学 2010

Examples9

Examples

Allen TD amp Rush MC (1998) The effects of organizational citizenship behavior on performance j d t A fi ld t d d l b t i tjudgments A field study and a laboratory experiment Journal of Applied Psychology 83(2) 247-260

Kenneth Law 同济大学 2010

10

AbAbstract

The process of linking organizational citizenship behavior (OCB) with performance judgments was investigated in a field and a laboratory study In the field study managers rated the task performance and OCB of 148 subordinates In the laboratory research 136 students viewed and ratedsubordinates In the laboratory research 136 students viewed and rated videotaped segments of teaching performance that demonstrated either high or low task performance and high or low OCB

Citizenship Behaviors Overall evaluationBehaviors

Kenneth Law 同济大学 2010

11

High OCB Low OCB

Manipulation

Results

Performance rating Performance ratingBy students (X1) gtgtgt By students (X2)

Kenneth Law 同济大学 2010

Types of experimental designs12

Types of experimental designs

1 Experimental designs

2Quasi experimentsbull Field experiments bull Time series designs

Kenneth Law 同济大学 2010

Experimental designs13

Experimental designsPretestposttest control-group designeg having one group of employees taking the training program in week 1 The second group which will get the training after group one is used as the control group

Why do we need a control group if we have pretest and posttest alreadyWhats wrong with just having posttest with control groups

XT1 T2Experimental group 1 2Experimental group

Control group T1 T2

Kenneth Law 同济大学 2010

Quasi experiment14

Quasi-experiment

- Experimental design without real manipulations- No randomization

- Hui C Lam SK amp Law KS (2000) Instrumental values of organizational citizenship behavior for promotion A field quasi-experiment Journal of Applied Psychology 85 822-828

Kenneth Law 同济大学 2010

15

Employees

P ti OCB High OCB Promotion OCB

Antecedents

Low OCB No Promotion OCB

Kenneth Law 同济大学 2010

Correlational surveys16

Correlational surveys

Predictor variables (independent variables)Criterion variables (dependent variables)

Research questionHow would a set of predictor variables affect a set of outcome

variables (causality)

Performance Promotion

time 1 time 1

Kenneth Law 同济大学 2010

Examples17

Barling J LughlinC amp Kelloway EK (2002) Development and test of a model linking safety-specific transformational leadership and occupational safety Journal of Applied Psychology 87(3) 488-496

Vancouver JB Thompson CM Tischner EC amp Putka DJ (2002) Two studies examining the negative effect of self-efficacy on performance Journal of Applied Psychology y p J f App y gy87(3) 506-516

Dvir T Eden D Avoilio BJ Shamir B (2002) ImpactDvir T Eden D Avoilio BJ Shamir B (2002) Impact of transformational leadership on follower development and performance A field experiment Academy of Management Journal 45(4) 735-744

Kenneth Law 同济大学 2010

Journal 45(4) 735-744

Survey study18

Survey study

rSafety‐specific

Occupational

rxy

transformational leadership

Occupational safety

Self‐efficacy Job performance

Transformational leadership

Job performance

Kenneth Law 同济大学 2010

Meta Analysis19

Meta Analysis

R h tiResearch questionCan we combine quantitative research in the past and summarize the research findings quantitatively

Judge T Heller D Mount M (2002) Five-Factor g ( )model of personality and job satisfaction A meta-analysis Journal of Applied Psychology 87(3) 530-541

Kenneth Law 同济大学 2010

Meta analysis20

y

Big‐5 personality individualism neuroticism conscientiousness agreeableness openness to experiences

Rxy is the correlation between agreeableness and job satisfaction

Study N rxy

1 150 35 2 247 ‐07

3 386 174 124 485 278 025 76 450 29

Kenneth Law 同济大学 2010

21

Kenneth Law 同济大学 2010

Issues in survey design 22

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Issues in survey design 23

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

1 What is your research question24

1 What is your research question Is the research question testableq

改善我国旅游业未来发展的几个意见

Are the constructs well defined 「企业进取性」和企业业绩表现的关系

Do we have validated scales to measure the constructs Existing scales Western scales

Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础

Kenneth Law 同济大学 2010

What is the research question25

qIs EQ really useful in predicting work outcomes

Would a firm using a strategic human resource management approach be more competitive

What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC

Can supervisors distinguish task performance from contextual performance in the PRC

What are the antecedents of contextual performance

Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC

Kenneth Law 同济大学 2010

生涯管理要素圖26

生涯情況 信息 計劃 資源目標

探索的動機 候迭人的信息

生活的目標

1個人戰略2時間

1 解決個人問題的技能

2 控制1自我評估價值技術興趣經驗

2組織評估表現潛力分配計劃

生涯發展目標 生涯計劃執行

分配計劃

現有內部勞資市場1工作調整需要2生涯之路結構3內部提升

生涯機會信息1生涯信息系統2生涯咨詢

1未來組織經濟目標

2未來所需職工

1組織人才資源發展戰略

2重要職務分配

1工作機會2贊助人3生涯管理者

Kenneth Law 同济大学 2010

Th di ti l f LMX

27

The mediating role of LMX

OCB

TransformationalLeadership LMX-MDM

OCB

77

3280

TaskPerformance

16

Kenneth Law 同济大学 2010

Contributions28

Contributions1 Theoretical contributions

bull New variables based on theorybull New perspectivesbull New findings

h d l l b2 Methodological contributionsbull New measures (eg new scale development)

N th d ( l l h)bull New methods (eg cross‐level research)

Kenneth Law 同济大学 2010

Identifying Research Questions29

Identifying Research Questions

员 企业的交换契约

公平理论与员工公民行为

Literature

员工企业的交换契约Incremental theoretical

contribution

Research Questions

M h d l l Practitioners公司治理

Methodological rigor

59岁现象

合资企业员工本土化

Kenneth Law 同济大学 2010

What is a research question30

What is a research question

My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust

How do you like that

Expected returnknowledge sharing

knowledge sharing

Benevolence

Expected return

Trust

s a gtendency

s a gbehaviors

Kenneth Law 同济大学 2010

A conversation hellip31

A conversation hellip

Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate

ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic

ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal

Kenneth Law 同济大学 2010

A different view32

A different view

We do research because we want to know We do research because we want to know the answer hellip

Therefore before we start a study we must have a research question

We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question

Kenneth Law 同济大学 2010

A different view33

A different view

With a research question you would automatically ask for a theoryautomatically ask for a theory

B d h ld Based on your theory you would automatically develop some hypotheses

Kenneth Law 同济大学 2010

The research process34

The research process

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Trusttendency behaviors

Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g

Kenneth Law 同济大学 2010

The research process35

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

H d k th t k l d h i i t ti i

Trust

How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g

This is because helliphellip (a theory)

Kenneth Law 同济大学 2010

The research process36

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Y d th k l d h i i t ti b

Trust

You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)

Kenneth Law 同济大学 2010

The research process37

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

(1) Some people have an in‐born tendency to share with

Trust

(1) Some people have an in‐born tendency to share with others

(2) Some people would expect reciprocal sharing in the future

(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)

(4) helliphellip

Kenneth Law 同济大学 2010

(4) helliphellip

Some questions38

1 Has anyone done anything on this topic before (literature review)

2 Which one is the most important reason to explain knowledge sharing (theory)

3 Are there established management theories that can explain the phenomenon (theory)

4 Are these antecedents or causes (theory)

5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)

6 What is the most appropriate research design to answer my research questionresearch question

Kenneth Law 同济大学 2010

When to use which design39

When to use which design

1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis

Kenneth Law 同济大学 2010

Do Not start your research40

Do Not start your research helliphellip

With some hypothesized relationships

With a model

Just because no one has done that before

Only because you see a significant correlation in your datasetyour dataset

Without looking at the literature

Without a research questionq

If you do not find it interesting

If you do not believe it is important

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

1 Qualitative Reviews3

1 Qualitative Reviews

C R iContent ReviewTheory buildingHypotheses developmentHypotheses development

Manor B (2002) International assignmentsManor B (2002) International assignments for career building A model of agency relationships and psychological contracts Academy of Management Review 27(3) 373Academy of Management Review 27(3) 373-391

Kenneth Law 同济大学 2010

Abstract4

Abstract

We present a theoretical model of international assignmentsin which we examine the alignment or nonalignment of the organizationrsquos and individualrsquos expectations of an assignmentorganization s and individual s expectations of an assignment and its effect on assignment success We consider four basic configurations in our model ndash mutual loyalty mutual transaction agent opportunism and principal opportunism ndashtransaction agent opportunism and principal opportunismand present a matrix of organization-individual alignment to help predict varying degrees of success in expatriate assignment and in repatriation Finally we apply this matrixassignment and in repatriation Finally we apply this matrix as a framework for understanding changes in the employment contract over time

Kenneth Law 同济大学 2010

The Model (Agency Theory)5

Relational Transactional

Individual

Cell I Mutual loyalty Cell II Agent opportunism

High organizational success in expatriation and repatriation (P1)Hi h i di id l i t i ti

Moderate organizational success in expatriation but failure in repatriation (P3)i h i di id l i i iel

ationa

l

ganiza

tion

Cell III Principal opportunism Cell IV Mutual transaction

High individual success in expatriation and repatriation (P2)

High individual success in expatriation but mixed success in repatriation (P4)

Re

Org

Moderate organizational success in expatriation and low success in repatriation (P5)Moderate individual success in

Moderate to high organizational success in expatriation and a better chance of organizational success in repatriation than that in the case of misalignment (P7)Moderate to high individual success in ns

action

al

expatriation but failure in repatriation (P6)

Moderate to high individual success in expatriation and a better chance of individual success in repatriation than that in the case of misalignment (P8)

Tra

Kenneth Law 同济大学 2010

2 Qualitative Studies6

2 Qualitative Studies

S tt R I (1991) M i t i i b t dSutton RI (1991) Maintaining norms about expressed emotions The case of bill collectors Administrative Science Quarterly 36(2) 245-269

Kenneth Law 同济大学 2010

7

AbstractA qualitative study of a bill-collection organization is used to identify norms about the emotions that collectors are expected to convey to debtors and theabout the emotions that collectors are expected to convey to debtors and the means used by the organization to maintain such norms given that collectors expressed emotions are simultaneously influenced by their inner feelings The data indicate that collectors are selected socialized and rewarded for following the general norm of conveying urgency (high arousal with a hint offollowing the general norm of conveying urgency (high arousal with a hint of irritation) to debtors Collectors are further socialized and rewarded to adjust their expressed emotions in response to variations in debtor demeanor These contingent norms sometimes clash with collectors feelings toward debtors Bill collectors are taught to cope with such emotive dissonance by using cognitivecollectors are taught to cope with such emotive dissonance by using cognitive appraisals that help them become emotionally detached from debtors and by releasing unpleasant feelings without communicating these emotions to debtors

Kenneth Law 同济大学 2010

3 Laboratory designs8

3 Laboratory designs

Manipulation variables (independent variables)Manipulation variables (independent variables)Outcome variables (dependent variables)

Research questionHow would a set of outcome variables change as a result of our manipulation of some causalas a result of our manipulation of some causal variables (causality) lightning and thundering

Leadership style Employee performance

Kenneth Law 同济大学 2010

Examples9

Examples

Allen TD amp Rush MC (1998) The effects of organizational citizenship behavior on performance j d t A fi ld t d d l b t i tjudgments A field study and a laboratory experiment Journal of Applied Psychology 83(2) 247-260

Kenneth Law 同济大学 2010

10

AbAbstract

The process of linking organizational citizenship behavior (OCB) with performance judgments was investigated in a field and a laboratory study In the field study managers rated the task performance and OCB of 148 subordinates In the laboratory research 136 students viewed and ratedsubordinates In the laboratory research 136 students viewed and rated videotaped segments of teaching performance that demonstrated either high or low task performance and high or low OCB

Citizenship Behaviors Overall evaluationBehaviors

Kenneth Law 同济大学 2010

11

High OCB Low OCB

Manipulation

Results

Performance rating Performance ratingBy students (X1) gtgtgt By students (X2)

Kenneth Law 同济大学 2010

Types of experimental designs12

Types of experimental designs

1 Experimental designs

2Quasi experimentsbull Field experiments bull Time series designs

Kenneth Law 同济大学 2010

Experimental designs13

Experimental designsPretestposttest control-group designeg having one group of employees taking the training program in week 1 The second group which will get the training after group one is used as the control group

Why do we need a control group if we have pretest and posttest alreadyWhats wrong with just having posttest with control groups

XT1 T2Experimental group 1 2Experimental group

Control group T1 T2

Kenneth Law 同济大学 2010

Quasi experiment14

Quasi-experiment

- Experimental design without real manipulations- No randomization

- Hui C Lam SK amp Law KS (2000) Instrumental values of organizational citizenship behavior for promotion A field quasi-experiment Journal of Applied Psychology 85 822-828

Kenneth Law 同济大学 2010

15

Employees

P ti OCB High OCB Promotion OCB

Antecedents

Low OCB No Promotion OCB

Kenneth Law 同济大学 2010

Correlational surveys16

Correlational surveys

Predictor variables (independent variables)Criterion variables (dependent variables)

Research questionHow would a set of predictor variables affect a set of outcome

variables (causality)

Performance Promotion

time 1 time 1

Kenneth Law 同济大学 2010

Examples17

Barling J LughlinC amp Kelloway EK (2002) Development and test of a model linking safety-specific transformational leadership and occupational safety Journal of Applied Psychology 87(3) 488-496

Vancouver JB Thompson CM Tischner EC amp Putka DJ (2002) Two studies examining the negative effect of self-efficacy on performance Journal of Applied Psychology y p J f App y gy87(3) 506-516

Dvir T Eden D Avoilio BJ Shamir B (2002) ImpactDvir T Eden D Avoilio BJ Shamir B (2002) Impact of transformational leadership on follower development and performance A field experiment Academy of Management Journal 45(4) 735-744

Kenneth Law 同济大学 2010

Journal 45(4) 735-744

Survey study18

Survey study

rSafety‐specific

Occupational

rxy

transformational leadership

Occupational safety

Self‐efficacy Job performance

Transformational leadership

Job performance

Kenneth Law 同济大学 2010

Meta Analysis19

Meta Analysis

R h tiResearch questionCan we combine quantitative research in the past and summarize the research findings quantitatively

Judge T Heller D Mount M (2002) Five-Factor g ( )model of personality and job satisfaction A meta-analysis Journal of Applied Psychology 87(3) 530-541

Kenneth Law 同济大学 2010

Meta analysis20

y

Big‐5 personality individualism neuroticism conscientiousness agreeableness openness to experiences

Rxy is the correlation between agreeableness and job satisfaction

Study N rxy

1 150 35 2 247 ‐07

3 386 174 124 485 278 025 76 450 29

Kenneth Law 同济大学 2010

21

Kenneth Law 同济大学 2010

Issues in survey design 22

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Issues in survey design 23

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

1 What is your research question24

1 What is your research question Is the research question testableq

改善我国旅游业未来发展的几个意见

Are the constructs well defined 「企业进取性」和企业业绩表现的关系

Do we have validated scales to measure the constructs Existing scales Western scales

Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础

Kenneth Law 同济大学 2010

What is the research question25

qIs EQ really useful in predicting work outcomes

Would a firm using a strategic human resource management approach be more competitive

What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC

Can supervisors distinguish task performance from contextual performance in the PRC

What are the antecedents of contextual performance

Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC

Kenneth Law 同济大学 2010

生涯管理要素圖26

生涯情況 信息 計劃 資源目標

探索的動機 候迭人的信息

生活的目標

1個人戰略2時間

1 解決個人問題的技能

2 控制1自我評估價值技術興趣經驗

2組織評估表現潛力分配計劃

生涯發展目標 生涯計劃執行

分配計劃

現有內部勞資市場1工作調整需要2生涯之路結構3內部提升

生涯機會信息1生涯信息系統2生涯咨詢

1未來組織經濟目標

2未來所需職工

1組織人才資源發展戰略

2重要職務分配

1工作機會2贊助人3生涯管理者

Kenneth Law 同济大学 2010

Th di ti l f LMX

27

The mediating role of LMX

OCB

TransformationalLeadership LMX-MDM

OCB

77

3280

TaskPerformance

16

Kenneth Law 同济大学 2010

Contributions28

Contributions1 Theoretical contributions

bull New variables based on theorybull New perspectivesbull New findings

h d l l b2 Methodological contributionsbull New measures (eg new scale development)

N th d ( l l h)bull New methods (eg cross‐level research)

Kenneth Law 同济大学 2010

Identifying Research Questions29

Identifying Research Questions

员 企业的交换契约

公平理论与员工公民行为

Literature

员工企业的交换契约Incremental theoretical

contribution

Research Questions

M h d l l Practitioners公司治理

Methodological rigor

59岁现象

合资企业员工本土化

Kenneth Law 同济大学 2010

What is a research question30

What is a research question

My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust

How do you like that

Expected returnknowledge sharing

knowledge sharing

Benevolence

Expected return

Trust

s a gtendency

s a gbehaviors

Kenneth Law 同济大学 2010

A conversation hellip31

A conversation hellip

Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate

ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic

ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal

Kenneth Law 同济大学 2010

A different view32

A different view

We do research because we want to know We do research because we want to know the answer hellip

Therefore before we start a study we must have a research question

We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question

Kenneth Law 同济大学 2010

A different view33

A different view

With a research question you would automatically ask for a theoryautomatically ask for a theory

B d h ld Based on your theory you would automatically develop some hypotheses

Kenneth Law 同济大学 2010

The research process34

The research process

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Trusttendency behaviors

Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g

Kenneth Law 同济大学 2010

The research process35

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

H d k th t k l d h i i t ti i

Trust

How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g

This is because helliphellip (a theory)

Kenneth Law 同济大学 2010

The research process36

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Y d th k l d h i i t ti b

Trust

You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)

Kenneth Law 同济大学 2010

The research process37

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

(1) Some people have an in‐born tendency to share with

Trust

(1) Some people have an in‐born tendency to share with others

(2) Some people would expect reciprocal sharing in the future

(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)

(4) helliphellip

Kenneth Law 同济大学 2010

(4) helliphellip

Some questions38

1 Has anyone done anything on this topic before (literature review)

2 Which one is the most important reason to explain knowledge sharing (theory)

3 Are there established management theories that can explain the phenomenon (theory)

4 Are these antecedents or causes (theory)

5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)

6 What is the most appropriate research design to answer my research questionresearch question

Kenneth Law 同济大学 2010

When to use which design39

When to use which design

1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis

Kenneth Law 同济大学 2010

Do Not start your research40

Do Not start your research helliphellip

With some hypothesized relationships

With a model

Just because no one has done that before

Only because you see a significant correlation in your datasetyour dataset

Without looking at the literature

Without a research questionq

If you do not find it interesting

If you do not believe it is important

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Abstract4

Abstract

We present a theoretical model of international assignmentsin which we examine the alignment or nonalignment of the organizationrsquos and individualrsquos expectations of an assignmentorganization s and individual s expectations of an assignment and its effect on assignment success We consider four basic configurations in our model ndash mutual loyalty mutual transaction agent opportunism and principal opportunism ndashtransaction agent opportunism and principal opportunismand present a matrix of organization-individual alignment to help predict varying degrees of success in expatriate assignment and in repatriation Finally we apply this matrixassignment and in repatriation Finally we apply this matrix as a framework for understanding changes in the employment contract over time

Kenneth Law 同济大学 2010

The Model (Agency Theory)5

Relational Transactional

Individual

Cell I Mutual loyalty Cell II Agent opportunism

High organizational success in expatriation and repatriation (P1)Hi h i di id l i t i ti

Moderate organizational success in expatriation but failure in repatriation (P3)i h i di id l i i iel

ationa

l

ganiza

tion

Cell III Principal opportunism Cell IV Mutual transaction

High individual success in expatriation and repatriation (P2)

High individual success in expatriation but mixed success in repatriation (P4)

Re

Org

Moderate organizational success in expatriation and low success in repatriation (P5)Moderate individual success in

Moderate to high organizational success in expatriation and a better chance of organizational success in repatriation than that in the case of misalignment (P7)Moderate to high individual success in ns

action

al

expatriation but failure in repatriation (P6)

Moderate to high individual success in expatriation and a better chance of individual success in repatriation than that in the case of misalignment (P8)

Tra

Kenneth Law 同济大学 2010

2 Qualitative Studies6

2 Qualitative Studies

S tt R I (1991) M i t i i b t dSutton RI (1991) Maintaining norms about expressed emotions The case of bill collectors Administrative Science Quarterly 36(2) 245-269

Kenneth Law 同济大学 2010

7

AbstractA qualitative study of a bill-collection organization is used to identify norms about the emotions that collectors are expected to convey to debtors and theabout the emotions that collectors are expected to convey to debtors and the means used by the organization to maintain such norms given that collectors expressed emotions are simultaneously influenced by their inner feelings The data indicate that collectors are selected socialized and rewarded for following the general norm of conveying urgency (high arousal with a hint offollowing the general norm of conveying urgency (high arousal with a hint of irritation) to debtors Collectors are further socialized and rewarded to adjust their expressed emotions in response to variations in debtor demeanor These contingent norms sometimes clash with collectors feelings toward debtors Bill collectors are taught to cope with such emotive dissonance by using cognitivecollectors are taught to cope with such emotive dissonance by using cognitive appraisals that help them become emotionally detached from debtors and by releasing unpleasant feelings without communicating these emotions to debtors

Kenneth Law 同济大学 2010

3 Laboratory designs8

3 Laboratory designs

Manipulation variables (independent variables)Manipulation variables (independent variables)Outcome variables (dependent variables)

Research questionHow would a set of outcome variables change as a result of our manipulation of some causalas a result of our manipulation of some causal variables (causality) lightning and thundering

Leadership style Employee performance

Kenneth Law 同济大学 2010

Examples9

Examples

Allen TD amp Rush MC (1998) The effects of organizational citizenship behavior on performance j d t A fi ld t d d l b t i tjudgments A field study and a laboratory experiment Journal of Applied Psychology 83(2) 247-260

Kenneth Law 同济大学 2010

10

AbAbstract

The process of linking organizational citizenship behavior (OCB) with performance judgments was investigated in a field and a laboratory study In the field study managers rated the task performance and OCB of 148 subordinates In the laboratory research 136 students viewed and ratedsubordinates In the laboratory research 136 students viewed and rated videotaped segments of teaching performance that demonstrated either high or low task performance and high or low OCB

Citizenship Behaviors Overall evaluationBehaviors

Kenneth Law 同济大学 2010

11

High OCB Low OCB

Manipulation

Results

Performance rating Performance ratingBy students (X1) gtgtgt By students (X2)

Kenneth Law 同济大学 2010

Types of experimental designs12

Types of experimental designs

1 Experimental designs

2Quasi experimentsbull Field experiments bull Time series designs

Kenneth Law 同济大学 2010

Experimental designs13

Experimental designsPretestposttest control-group designeg having one group of employees taking the training program in week 1 The second group which will get the training after group one is used as the control group

Why do we need a control group if we have pretest and posttest alreadyWhats wrong with just having posttest with control groups

XT1 T2Experimental group 1 2Experimental group

Control group T1 T2

Kenneth Law 同济大学 2010

Quasi experiment14

Quasi-experiment

- Experimental design without real manipulations- No randomization

- Hui C Lam SK amp Law KS (2000) Instrumental values of organizational citizenship behavior for promotion A field quasi-experiment Journal of Applied Psychology 85 822-828

Kenneth Law 同济大学 2010

15

Employees

P ti OCB High OCB Promotion OCB

Antecedents

Low OCB No Promotion OCB

Kenneth Law 同济大学 2010

Correlational surveys16

Correlational surveys

Predictor variables (independent variables)Criterion variables (dependent variables)

Research questionHow would a set of predictor variables affect a set of outcome

variables (causality)

Performance Promotion

time 1 time 1

Kenneth Law 同济大学 2010

Examples17

Barling J LughlinC amp Kelloway EK (2002) Development and test of a model linking safety-specific transformational leadership and occupational safety Journal of Applied Psychology 87(3) 488-496

Vancouver JB Thompson CM Tischner EC amp Putka DJ (2002) Two studies examining the negative effect of self-efficacy on performance Journal of Applied Psychology y p J f App y gy87(3) 506-516

Dvir T Eden D Avoilio BJ Shamir B (2002) ImpactDvir T Eden D Avoilio BJ Shamir B (2002) Impact of transformational leadership on follower development and performance A field experiment Academy of Management Journal 45(4) 735-744

Kenneth Law 同济大学 2010

Journal 45(4) 735-744

Survey study18

Survey study

rSafety‐specific

Occupational

rxy

transformational leadership

Occupational safety

Self‐efficacy Job performance

Transformational leadership

Job performance

Kenneth Law 同济大学 2010

Meta Analysis19

Meta Analysis

R h tiResearch questionCan we combine quantitative research in the past and summarize the research findings quantitatively

Judge T Heller D Mount M (2002) Five-Factor g ( )model of personality and job satisfaction A meta-analysis Journal of Applied Psychology 87(3) 530-541

Kenneth Law 同济大学 2010

Meta analysis20

y

Big‐5 personality individualism neuroticism conscientiousness agreeableness openness to experiences

Rxy is the correlation between agreeableness and job satisfaction

Study N rxy

1 150 35 2 247 ‐07

3 386 174 124 485 278 025 76 450 29

Kenneth Law 同济大学 2010

21

Kenneth Law 同济大学 2010

Issues in survey design 22

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Issues in survey design 23

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

1 What is your research question24

1 What is your research question Is the research question testableq

改善我国旅游业未来发展的几个意见

Are the constructs well defined 「企业进取性」和企业业绩表现的关系

Do we have validated scales to measure the constructs Existing scales Western scales

Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础

Kenneth Law 同济大学 2010

What is the research question25

qIs EQ really useful in predicting work outcomes

Would a firm using a strategic human resource management approach be more competitive

What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC

Can supervisors distinguish task performance from contextual performance in the PRC

What are the antecedents of contextual performance

Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC

Kenneth Law 同济大学 2010

生涯管理要素圖26

生涯情況 信息 計劃 資源目標

探索的動機 候迭人的信息

生活的目標

1個人戰略2時間

1 解決個人問題的技能

2 控制1自我評估價值技術興趣經驗

2組織評估表現潛力分配計劃

生涯發展目標 生涯計劃執行

分配計劃

現有內部勞資市場1工作調整需要2生涯之路結構3內部提升

生涯機會信息1生涯信息系統2生涯咨詢

1未來組織經濟目標

2未來所需職工

1組織人才資源發展戰略

2重要職務分配

1工作機會2贊助人3生涯管理者

Kenneth Law 同济大学 2010

Th di ti l f LMX

27

The mediating role of LMX

OCB

TransformationalLeadership LMX-MDM

OCB

77

3280

TaskPerformance

16

Kenneth Law 同济大学 2010

Contributions28

Contributions1 Theoretical contributions

bull New variables based on theorybull New perspectivesbull New findings

h d l l b2 Methodological contributionsbull New measures (eg new scale development)

N th d ( l l h)bull New methods (eg cross‐level research)

Kenneth Law 同济大学 2010

Identifying Research Questions29

Identifying Research Questions

员 企业的交换契约

公平理论与员工公民行为

Literature

员工企业的交换契约Incremental theoretical

contribution

Research Questions

M h d l l Practitioners公司治理

Methodological rigor

59岁现象

合资企业员工本土化

Kenneth Law 同济大学 2010

What is a research question30

What is a research question

My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust

How do you like that

Expected returnknowledge sharing

knowledge sharing

Benevolence

Expected return

Trust

s a gtendency

s a gbehaviors

Kenneth Law 同济大学 2010

A conversation hellip31

A conversation hellip

Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate

ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic

ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal

Kenneth Law 同济大学 2010

A different view32

A different view

We do research because we want to know We do research because we want to know the answer hellip

Therefore before we start a study we must have a research question

We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question

Kenneth Law 同济大学 2010

A different view33

A different view

With a research question you would automatically ask for a theoryautomatically ask for a theory

B d h ld Based on your theory you would automatically develop some hypotheses

Kenneth Law 同济大学 2010

The research process34

The research process

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Trusttendency behaviors

Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g

Kenneth Law 同济大学 2010

The research process35

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

H d k th t k l d h i i t ti i

Trust

How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g

This is because helliphellip (a theory)

Kenneth Law 同济大学 2010

The research process36

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Y d th k l d h i i t ti b

Trust

You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)

Kenneth Law 同济大学 2010

The research process37

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

(1) Some people have an in‐born tendency to share with

Trust

(1) Some people have an in‐born tendency to share with others

(2) Some people would expect reciprocal sharing in the future

(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)

(4) helliphellip

Kenneth Law 同济大学 2010

(4) helliphellip

Some questions38

1 Has anyone done anything on this topic before (literature review)

2 Which one is the most important reason to explain knowledge sharing (theory)

3 Are there established management theories that can explain the phenomenon (theory)

4 Are these antecedents or causes (theory)

5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)

6 What is the most appropriate research design to answer my research questionresearch question

Kenneth Law 同济大学 2010

When to use which design39

When to use which design

1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis

Kenneth Law 同济大学 2010

Do Not start your research40

Do Not start your research helliphellip

With some hypothesized relationships

With a model

Just because no one has done that before

Only because you see a significant correlation in your datasetyour dataset

Without looking at the literature

Without a research questionq

If you do not find it interesting

If you do not believe it is important

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

The Model (Agency Theory)5

Relational Transactional

Individual

Cell I Mutual loyalty Cell II Agent opportunism

High organizational success in expatriation and repatriation (P1)Hi h i di id l i t i ti

Moderate organizational success in expatriation but failure in repatriation (P3)i h i di id l i i iel

ationa

l

ganiza

tion

Cell III Principal opportunism Cell IV Mutual transaction

High individual success in expatriation and repatriation (P2)

High individual success in expatriation but mixed success in repatriation (P4)

Re

Org

Moderate organizational success in expatriation and low success in repatriation (P5)Moderate individual success in

Moderate to high organizational success in expatriation and a better chance of organizational success in repatriation than that in the case of misalignment (P7)Moderate to high individual success in ns

action

al

expatriation but failure in repatriation (P6)

Moderate to high individual success in expatriation and a better chance of individual success in repatriation than that in the case of misalignment (P8)

Tra

Kenneth Law 同济大学 2010

2 Qualitative Studies6

2 Qualitative Studies

S tt R I (1991) M i t i i b t dSutton RI (1991) Maintaining norms about expressed emotions The case of bill collectors Administrative Science Quarterly 36(2) 245-269

Kenneth Law 同济大学 2010

7

AbstractA qualitative study of a bill-collection organization is used to identify norms about the emotions that collectors are expected to convey to debtors and theabout the emotions that collectors are expected to convey to debtors and the means used by the organization to maintain such norms given that collectors expressed emotions are simultaneously influenced by their inner feelings The data indicate that collectors are selected socialized and rewarded for following the general norm of conveying urgency (high arousal with a hint offollowing the general norm of conveying urgency (high arousal with a hint of irritation) to debtors Collectors are further socialized and rewarded to adjust their expressed emotions in response to variations in debtor demeanor These contingent norms sometimes clash with collectors feelings toward debtors Bill collectors are taught to cope with such emotive dissonance by using cognitivecollectors are taught to cope with such emotive dissonance by using cognitive appraisals that help them become emotionally detached from debtors and by releasing unpleasant feelings without communicating these emotions to debtors

Kenneth Law 同济大学 2010

3 Laboratory designs8

3 Laboratory designs

Manipulation variables (independent variables)Manipulation variables (independent variables)Outcome variables (dependent variables)

Research questionHow would a set of outcome variables change as a result of our manipulation of some causalas a result of our manipulation of some causal variables (causality) lightning and thundering

Leadership style Employee performance

Kenneth Law 同济大学 2010

Examples9

Examples

Allen TD amp Rush MC (1998) The effects of organizational citizenship behavior on performance j d t A fi ld t d d l b t i tjudgments A field study and a laboratory experiment Journal of Applied Psychology 83(2) 247-260

Kenneth Law 同济大学 2010

10

AbAbstract

The process of linking organizational citizenship behavior (OCB) with performance judgments was investigated in a field and a laboratory study In the field study managers rated the task performance and OCB of 148 subordinates In the laboratory research 136 students viewed and ratedsubordinates In the laboratory research 136 students viewed and rated videotaped segments of teaching performance that demonstrated either high or low task performance and high or low OCB

Citizenship Behaviors Overall evaluationBehaviors

Kenneth Law 同济大学 2010

11

High OCB Low OCB

Manipulation

Results

Performance rating Performance ratingBy students (X1) gtgtgt By students (X2)

Kenneth Law 同济大学 2010

Types of experimental designs12

Types of experimental designs

1 Experimental designs

2Quasi experimentsbull Field experiments bull Time series designs

Kenneth Law 同济大学 2010

Experimental designs13

Experimental designsPretestposttest control-group designeg having one group of employees taking the training program in week 1 The second group which will get the training after group one is used as the control group

Why do we need a control group if we have pretest and posttest alreadyWhats wrong with just having posttest with control groups

XT1 T2Experimental group 1 2Experimental group

Control group T1 T2

Kenneth Law 同济大学 2010

Quasi experiment14

Quasi-experiment

- Experimental design without real manipulations- No randomization

- Hui C Lam SK amp Law KS (2000) Instrumental values of organizational citizenship behavior for promotion A field quasi-experiment Journal of Applied Psychology 85 822-828

Kenneth Law 同济大学 2010

15

Employees

P ti OCB High OCB Promotion OCB

Antecedents

Low OCB No Promotion OCB

Kenneth Law 同济大学 2010

Correlational surveys16

Correlational surveys

Predictor variables (independent variables)Criterion variables (dependent variables)

Research questionHow would a set of predictor variables affect a set of outcome

variables (causality)

Performance Promotion

time 1 time 1

Kenneth Law 同济大学 2010

Examples17

Barling J LughlinC amp Kelloway EK (2002) Development and test of a model linking safety-specific transformational leadership and occupational safety Journal of Applied Psychology 87(3) 488-496

Vancouver JB Thompson CM Tischner EC amp Putka DJ (2002) Two studies examining the negative effect of self-efficacy on performance Journal of Applied Psychology y p J f App y gy87(3) 506-516

Dvir T Eden D Avoilio BJ Shamir B (2002) ImpactDvir T Eden D Avoilio BJ Shamir B (2002) Impact of transformational leadership on follower development and performance A field experiment Academy of Management Journal 45(4) 735-744

Kenneth Law 同济大学 2010

Journal 45(4) 735-744

Survey study18

Survey study

rSafety‐specific

Occupational

rxy

transformational leadership

Occupational safety

Self‐efficacy Job performance

Transformational leadership

Job performance

Kenneth Law 同济大学 2010

Meta Analysis19

Meta Analysis

R h tiResearch questionCan we combine quantitative research in the past and summarize the research findings quantitatively

Judge T Heller D Mount M (2002) Five-Factor g ( )model of personality and job satisfaction A meta-analysis Journal of Applied Psychology 87(3) 530-541

Kenneth Law 同济大学 2010

Meta analysis20

y

Big‐5 personality individualism neuroticism conscientiousness agreeableness openness to experiences

Rxy is the correlation between agreeableness and job satisfaction

Study N rxy

1 150 35 2 247 ‐07

3 386 174 124 485 278 025 76 450 29

Kenneth Law 同济大学 2010

21

Kenneth Law 同济大学 2010

Issues in survey design 22

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Issues in survey design 23

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

1 What is your research question24

1 What is your research question Is the research question testableq

改善我国旅游业未来发展的几个意见

Are the constructs well defined 「企业进取性」和企业业绩表现的关系

Do we have validated scales to measure the constructs Existing scales Western scales

Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础

Kenneth Law 同济大学 2010

What is the research question25

qIs EQ really useful in predicting work outcomes

Would a firm using a strategic human resource management approach be more competitive

What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC

Can supervisors distinguish task performance from contextual performance in the PRC

What are the antecedents of contextual performance

Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC

Kenneth Law 同济大学 2010

生涯管理要素圖26

生涯情況 信息 計劃 資源目標

探索的動機 候迭人的信息

生活的目標

1個人戰略2時間

1 解決個人問題的技能

2 控制1自我評估價值技術興趣經驗

2組織評估表現潛力分配計劃

生涯發展目標 生涯計劃執行

分配計劃

現有內部勞資市場1工作調整需要2生涯之路結構3內部提升

生涯機會信息1生涯信息系統2生涯咨詢

1未來組織經濟目標

2未來所需職工

1組織人才資源發展戰略

2重要職務分配

1工作機會2贊助人3生涯管理者

Kenneth Law 同济大学 2010

Th di ti l f LMX

27

The mediating role of LMX

OCB

TransformationalLeadership LMX-MDM

OCB

77

3280

TaskPerformance

16

Kenneth Law 同济大学 2010

Contributions28

Contributions1 Theoretical contributions

bull New variables based on theorybull New perspectivesbull New findings

h d l l b2 Methodological contributionsbull New measures (eg new scale development)

N th d ( l l h)bull New methods (eg cross‐level research)

Kenneth Law 同济大学 2010

Identifying Research Questions29

Identifying Research Questions

员 企业的交换契约

公平理论与员工公民行为

Literature

员工企业的交换契约Incremental theoretical

contribution

Research Questions

M h d l l Practitioners公司治理

Methodological rigor

59岁现象

合资企业员工本土化

Kenneth Law 同济大学 2010

What is a research question30

What is a research question

My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust

How do you like that

Expected returnknowledge sharing

knowledge sharing

Benevolence

Expected return

Trust

s a gtendency

s a gbehaviors

Kenneth Law 同济大学 2010

A conversation hellip31

A conversation hellip

Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate

ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic

ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal

Kenneth Law 同济大学 2010

A different view32

A different view

We do research because we want to know We do research because we want to know the answer hellip

Therefore before we start a study we must have a research question

We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question

Kenneth Law 同济大学 2010

A different view33

A different view

With a research question you would automatically ask for a theoryautomatically ask for a theory

B d h ld Based on your theory you would automatically develop some hypotheses

Kenneth Law 同济大学 2010

The research process34

The research process

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Trusttendency behaviors

Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g

Kenneth Law 同济大学 2010

The research process35

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

H d k th t k l d h i i t ti i

Trust

How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g

This is because helliphellip (a theory)

Kenneth Law 同济大学 2010

The research process36

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Y d th k l d h i i t ti b

Trust

You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)

Kenneth Law 同济大学 2010

The research process37

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

(1) Some people have an in‐born tendency to share with

Trust

(1) Some people have an in‐born tendency to share with others

(2) Some people would expect reciprocal sharing in the future

(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)

(4) helliphellip

Kenneth Law 同济大学 2010

(4) helliphellip

Some questions38

1 Has anyone done anything on this topic before (literature review)

2 Which one is the most important reason to explain knowledge sharing (theory)

3 Are there established management theories that can explain the phenomenon (theory)

4 Are these antecedents or causes (theory)

5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)

6 What is the most appropriate research design to answer my research questionresearch question

Kenneth Law 同济大学 2010

When to use which design39

When to use which design

1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis

Kenneth Law 同济大学 2010

Do Not start your research40

Do Not start your research helliphellip

With some hypothesized relationships

With a model

Just because no one has done that before

Only because you see a significant correlation in your datasetyour dataset

Without looking at the literature

Without a research questionq

If you do not find it interesting

If you do not believe it is important

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

2 Qualitative Studies6

2 Qualitative Studies

S tt R I (1991) M i t i i b t dSutton RI (1991) Maintaining norms about expressed emotions The case of bill collectors Administrative Science Quarterly 36(2) 245-269

Kenneth Law 同济大学 2010

7

AbstractA qualitative study of a bill-collection organization is used to identify norms about the emotions that collectors are expected to convey to debtors and theabout the emotions that collectors are expected to convey to debtors and the means used by the organization to maintain such norms given that collectors expressed emotions are simultaneously influenced by their inner feelings The data indicate that collectors are selected socialized and rewarded for following the general norm of conveying urgency (high arousal with a hint offollowing the general norm of conveying urgency (high arousal with a hint of irritation) to debtors Collectors are further socialized and rewarded to adjust their expressed emotions in response to variations in debtor demeanor These contingent norms sometimes clash with collectors feelings toward debtors Bill collectors are taught to cope with such emotive dissonance by using cognitivecollectors are taught to cope with such emotive dissonance by using cognitive appraisals that help them become emotionally detached from debtors and by releasing unpleasant feelings without communicating these emotions to debtors

Kenneth Law 同济大学 2010

3 Laboratory designs8

3 Laboratory designs

Manipulation variables (independent variables)Manipulation variables (independent variables)Outcome variables (dependent variables)

Research questionHow would a set of outcome variables change as a result of our manipulation of some causalas a result of our manipulation of some causal variables (causality) lightning and thundering

Leadership style Employee performance

Kenneth Law 同济大学 2010

Examples9

Examples

Allen TD amp Rush MC (1998) The effects of organizational citizenship behavior on performance j d t A fi ld t d d l b t i tjudgments A field study and a laboratory experiment Journal of Applied Psychology 83(2) 247-260

Kenneth Law 同济大学 2010

10

AbAbstract

The process of linking organizational citizenship behavior (OCB) with performance judgments was investigated in a field and a laboratory study In the field study managers rated the task performance and OCB of 148 subordinates In the laboratory research 136 students viewed and ratedsubordinates In the laboratory research 136 students viewed and rated videotaped segments of teaching performance that demonstrated either high or low task performance and high or low OCB

Citizenship Behaviors Overall evaluationBehaviors

Kenneth Law 同济大学 2010

11

High OCB Low OCB

Manipulation

Results

Performance rating Performance ratingBy students (X1) gtgtgt By students (X2)

Kenneth Law 同济大学 2010

Types of experimental designs12

Types of experimental designs

1 Experimental designs

2Quasi experimentsbull Field experiments bull Time series designs

Kenneth Law 同济大学 2010

Experimental designs13

Experimental designsPretestposttest control-group designeg having one group of employees taking the training program in week 1 The second group which will get the training after group one is used as the control group

Why do we need a control group if we have pretest and posttest alreadyWhats wrong with just having posttest with control groups

XT1 T2Experimental group 1 2Experimental group

Control group T1 T2

Kenneth Law 同济大学 2010

Quasi experiment14

Quasi-experiment

- Experimental design without real manipulations- No randomization

- Hui C Lam SK amp Law KS (2000) Instrumental values of organizational citizenship behavior for promotion A field quasi-experiment Journal of Applied Psychology 85 822-828

Kenneth Law 同济大学 2010

15

Employees

P ti OCB High OCB Promotion OCB

Antecedents

Low OCB No Promotion OCB

Kenneth Law 同济大学 2010

Correlational surveys16

Correlational surveys

Predictor variables (independent variables)Criterion variables (dependent variables)

Research questionHow would a set of predictor variables affect a set of outcome

variables (causality)

Performance Promotion

time 1 time 1

Kenneth Law 同济大学 2010

Examples17

Barling J LughlinC amp Kelloway EK (2002) Development and test of a model linking safety-specific transformational leadership and occupational safety Journal of Applied Psychology 87(3) 488-496

Vancouver JB Thompson CM Tischner EC amp Putka DJ (2002) Two studies examining the negative effect of self-efficacy on performance Journal of Applied Psychology y p J f App y gy87(3) 506-516

Dvir T Eden D Avoilio BJ Shamir B (2002) ImpactDvir T Eden D Avoilio BJ Shamir B (2002) Impact of transformational leadership on follower development and performance A field experiment Academy of Management Journal 45(4) 735-744

Kenneth Law 同济大学 2010

Journal 45(4) 735-744

Survey study18

Survey study

rSafety‐specific

Occupational

rxy

transformational leadership

Occupational safety

Self‐efficacy Job performance

Transformational leadership

Job performance

Kenneth Law 同济大学 2010

Meta Analysis19

Meta Analysis

R h tiResearch questionCan we combine quantitative research in the past and summarize the research findings quantitatively

Judge T Heller D Mount M (2002) Five-Factor g ( )model of personality and job satisfaction A meta-analysis Journal of Applied Psychology 87(3) 530-541

Kenneth Law 同济大学 2010

Meta analysis20

y

Big‐5 personality individualism neuroticism conscientiousness agreeableness openness to experiences

Rxy is the correlation between agreeableness and job satisfaction

Study N rxy

1 150 35 2 247 ‐07

3 386 174 124 485 278 025 76 450 29

Kenneth Law 同济大学 2010

21

Kenneth Law 同济大学 2010

Issues in survey design 22

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Issues in survey design 23

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

1 What is your research question24

1 What is your research question Is the research question testableq

改善我国旅游业未来发展的几个意见

Are the constructs well defined 「企业进取性」和企业业绩表现的关系

Do we have validated scales to measure the constructs Existing scales Western scales

Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础

Kenneth Law 同济大学 2010

What is the research question25

qIs EQ really useful in predicting work outcomes

Would a firm using a strategic human resource management approach be more competitive

What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC

Can supervisors distinguish task performance from contextual performance in the PRC

What are the antecedents of contextual performance

Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC

Kenneth Law 同济大学 2010

生涯管理要素圖26

生涯情況 信息 計劃 資源目標

探索的動機 候迭人的信息

生活的目標

1個人戰略2時間

1 解決個人問題的技能

2 控制1自我評估價值技術興趣經驗

2組織評估表現潛力分配計劃

生涯發展目標 生涯計劃執行

分配計劃

現有內部勞資市場1工作調整需要2生涯之路結構3內部提升

生涯機會信息1生涯信息系統2生涯咨詢

1未來組織經濟目標

2未來所需職工

1組織人才資源發展戰略

2重要職務分配

1工作機會2贊助人3生涯管理者

Kenneth Law 同济大学 2010

Th di ti l f LMX

27

The mediating role of LMX

OCB

TransformationalLeadership LMX-MDM

OCB

77

3280

TaskPerformance

16

Kenneth Law 同济大学 2010

Contributions28

Contributions1 Theoretical contributions

bull New variables based on theorybull New perspectivesbull New findings

h d l l b2 Methodological contributionsbull New measures (eg new scale development)

N th d ( l l h)bull New methods (eg cross‐level research)

Kenneth Law 同济大学 2010

Identifying Research Questions29

Identifying Research Questions

员 企业的交换契约

公平理论与员工公民行为

Literature

员工企业的交换契约Incremental theoretical

contribution

Research Questions

M h d l l Practitioners公司治理

Methodological rigor

59岁现象

合资企业员工本土化

Kenneth Law 同济大学 2010

What is a research question30

What is a research question

My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust

How do you like that

Expected returnknowledge sharing

knowledge sharing

Benevolence

Expected return

Trust

s a gtendency

s a gbehaviors

Kenneth Law 同济大学 2010

A conversation hellip31

A conversation hellip

Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate

ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic

ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal

Kenneth Law 同济大学 2010

A different view32

A different view

We do research because we want to know We do research because we want to know the answer hellip

Therefore before we start a study we must have a research question

We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question

Kenneth Law 同济大学 2010

A different view33

A different view

With a research question you would automatically ask for a theoryautomatically ask for a theory

B d h ld Based on your theory you would automatically develop some hypotheses

Kenneth Law 同济大学 2010

The research process34

The research process

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Trusttendency behaviors

Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g

Kenneth Law 同济大学 2010

The research process35

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

H d k th t k l d h i i t ti i

Trust

How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g

This is because helliphellip (a theory)

Kenneth Law 同济大学 2010

The research process36

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Y d th k l d h i i t ti b

Trust

You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)

Kenneth Law 同济大学 2010

The research process37

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

(1) Some people have an in‐born tendency to share with

Trust

(1) Some people have an in‐born tendency to share with others

(2) Some people would expect reciprocal sharing in the future

(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)

(4) helliphellip

Kenneth Law 同济大学 2010

(4) helliphellip

Some questions38

1 Has anyone done anything on this topic before (literature review)

2 Which one is the most important reason to explain knowledge sharing (theory)

3 Are there established management theories that can explain the phenomenon (theory)

4 Are these antecedents or causes (theory)

5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)

6 What is the most appropriate research design to answer my research questionresearch question

Kenneth Law 同济大学 2010

When to use which design39

When to use which design

1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis

Kenneth Law 同济大学 2010

Do Not start your research40

Do Not start your research helliphellip

With some hypothesized relationships

With a model

Just because no one has done that before

Only because you see a significant correlation in your datasetyour dataset

Without looking at the literature

Without a research questionq

If you do not find it interesting

If you do not believe it is important

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

7

AbstractA qualitative study of a bill-collection organization is used to identify norms about the emotions that collectors are expected to convey to debtors and theabout the emotions that collectors are expected to convey to debtors and the means used by the organization to maintain such norms given that collectors expressed emotions are simultaneously influenced by their inner feelings The data indicate that collectors are selected socialized and rewarded for following the general norm of conveying urgency (high arousal with a hint offollowing the general norm of conveying urgency (high arousal with a hint of irritation) to debtors Collectors are further socialized and rewarded to adjust their expressed emotions in response to variations in debtor demeanor These contingent norms sometimes clash with collectors feelings toward debtors Bill collectors are taught to cope with such emotive dissonance by using cognitivecollectors are taught to cope with such emotive dissonance by using cognitive appraisals that help them become emotionally detached from debtors and by releasing unpleasant feelings without communicating these emotions to debtors

Kenneth Law 同济大学 2010

3 Laboratory designs8

3 Laboratory designs

Manipulation variables (independent variables)Manipulation variables (independent variables)Outcome variables (dependent variables)

Research questionHow would a set of outcome variables change as a result of our manipulation of some causalas a result of our manipulation of some causal variables (causality) lightning and thundering

Leadership style Employee performance

Kenneth Law 同济大学 2010

Examples9

Examples

Allen TD amp Rush MC (1998) The effects of organizational citizenship behavior on performance j d t A fi ld t d d l b t i tjudgments A field study and a laboratory experiment Journal of Applied Psychology 83(2) 247-260

Kenneth Law 同济大学 2010

10

AbAbstract

The process of linking organizational citizenship behavior (OCB) with performance judgments was investigated in a field and a laboratory study In the field study managers rated the task performance and OCB of 148 subordinates In the laboratory research 136 students viewed and ratedsubordinates In the laboratory research 136 students viewed and rated videotaped segments of teaching performance that demonstrated either high or low task performance and high or low OCB

Citizenship Behaviors Overall evaluationBehaviors

Kenneth Law 同济大学 2010

11

High OCB Low OCB

Manipulation

Results

Performance rating Performance ratingBy students (X1) gtgtgt By students (X2)

Kenneth Law 同济大学 2010

Types of experimental designs12

Types of experimental designs

1 Experimental designs

2Quasi experimentsbull Field experiments bull Time series designs

Kenneth Law 同济大学 2010

Experimental designs13

Experimental designsPretestposttest control-group designeg having one group of employees taking the training program in week 1 The second group which will get the training after group one is used as the control group

Why do we need a control group if we have pretest and posttest alreadyWhats wrong with just having posttest with control groups

XT1 T2Experimental group 1 2Experimental group

Control group T1 T2

Kenneth Law 同济大学 2010

Quasi experiment14

Quasi-experiment

- Experimental design without real manipulations- No randomization

- Hui C Lam SK amp Law KS (2000) Instrumental values of organizational citizenship behavior for promotion A field quasi-experiment Journal of Applied Psychology 85 822-828

Kenneth Law 同济大学 2010

15

Employees

P ti OCB High OCB Promotion OCB

Antecedents

Low OCB No Promotion OCB

Kenneth Law 同济大学 2010

Correlational surveys16

Correlational surveys

Predictor variables (independent variables)Criterion variables (dependent variables)

Research questionHow would a set of predictor variables affect a set of outcome

variables (causality)

Performance Promotion

time 1 time 1

Kenneth Law 同济大学 2010

Examples17

Barling J LughlinC amp Kelloway EK (2002) Development and test of a model linking safety-specific transformational leadership and occupational safety Journal of Applied Psychology 87(3) 488-496

Vancouver JB Thompson CM Tischner EC amp Putka DJ (2002) Two studies examining the negative effect of self-efficacy on performance Journal of Applied Psychology y p J f App y gy87(3) 506-516

Dvir T Eden D Avoilio BJ Shamir B (2002) ImpactDvir T Eden D Avoilio BJ Shamir B (2002) Impact of transformational leadership on follower development and performance A field experiment Academy of Management Journal 45(4) 735-744

Kenneth Law 同济大学 2010

Journal 45(4) 735-744

Survey study18

Survey study

rSafety‐specific

Occupational

rxy

transformational leadership

Occupational safety

Self‐efficacy Job performance

Transformational leadership

Job performance

Kenneth Law 同济大学 2010

Meta Analysis19

Meta Analysis

R h tiResearch questionCan we combine quantitative research in the past and summarize the research findings quantitatively

Judge T Heller D Mount M (2002) Five-Factor g ( )model of personality and job satisfaction A meta-analysis Journal of Applied Psychology 87(3) 530-541

Kenneth Law 同济大学 2010

Meta analysis20

y

Big‐5 personality individualism neuroticism conscientiousness agreeableness openness to experiences

Rxy is the correlation between agreeableness and job satisfaction

Study N rxy

1 150 35 2 247 ‐07

3 386 174 124 485 278 025 76 450 29

Kenneth Law 同济大学 2010

21

Kenneth Law 同济大学 2010

Issues in survey design 22

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Issues in survey design 23

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

1 What is your research question24

1 What is your research question Is the research question testableq

改善我国旅游业未来发展的几个意见

Are the constructs well defined 「企业进取性」和企业业绩表现的关系

Do we have validated scales to measure the constructs Existing scales Western scales

Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础

Kenneth Law 同济大学 2010

What is the research question25

qIs EQ really useful in predicting work outcomes

Would a firm using a strategic human resource management approach be more competitive

What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC

Can supervisors distinguish task performance from contextual performance in the PRC

What are the antecedents of contextual performance

Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC

Kenneth Law 同济大学 2010

生涯管理要素圖26

生涯情況 信息 計劃 資源目標

探索的動機 候迭人的信息

生活的目標

1個人戰略2時間

1 解決個人問題的技能

2 控制1自我評估價值技術興趣經驗

2組織評估表現潛力分配計劃

生涯發展目標 生涯計劃執行

分配計劃

現有內部勞資市場1工作調整需要2生涯之路結構3內部提升

生涯機會信息1生涯信息系統2生涯咨詢

1未來組織經濟目標

2未來所需職工

1組織人才資源發展戰略

2重要職務分配

1工作機會2贊助人3生涯管理者

Kenneth Law 同济大学 2010

Th di ti l f LMX

27

The mediating role of LMX

OCB

TransformationalLeadership LMX-MDM

OCB

77

3280

TaskPerformance

16

Kenneth Law 同济大学 2010

Contributions28

Contributions1 Theoretical contributions

bull New variables based on theorybull New perspectivesbull New findings

h d l l b2 Methodological contributionsbull New measures (eg new scale development)

N th d ( l l h)bull New methods (eg cross‐level research)

Kenneth Law 同济大学 2010

Identifying Research Questions29

Identifying Research Questions

员 企业的交换契约

公平理论与员工公民行为

Literature

员工企业的交换契约Incremental theoretical

contribution

Research Questions

M h d l l Practitioners公司治理

Methodological rigor

59岁现象

合资企业员工本土化

Kenneth Law 同济大学 2010

What is a research question30

What is a research question

My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust

How do you like that

Expected returnknowledge sharing

knowledge sharing

Benevolence

Expected return

Trust

s a gtendency

s a gbehaviors

Kenneth Law 同济大学 2010

A conversation hellip31

A conversation hellip

Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate

ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic

ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal

Kenneth Law 同济大学 2010

A different view32

A different view

We do research because we want to know We do research because we want to know the answer hellip

Therefore before we start a study we must have a research question

We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question

Kenneth Law 同济大学 2010

A different view33

A different view

With a research question you would automatically ask for a theoryautomatically ask for a theory

B d h ld Based on your theory you would automatically develop some hypotheses

Kenneth Law 同济大学 2010

The research process34

The research process

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Trusttendency behaviors

Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g

Kenneth Law 同济大学 2010

The research process35

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

H d k th t k l d h i i t ti i

Trust

How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g

This is because helliphellip (a theory)

Kenneth Law 同济大学 2010

The research process36

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Y d th k l d h i i t ti b

Trust

You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)

Kenneth Law 同济大学 2010

The research process37

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

(1) Some people have an in‐born tendency to share with

Trust

(1) Some people have an in‐born tendency to share with others

(2) Some people would expect reciprocal sharing in the future

(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)

(4) helliphellip

Kenneth Law 同济大学 2010

(4) helliphellip

Some questions38

1 Has anyone done anything on this topic before (literature review)

2 Which one is the most important reason to explain knowledge sharing (theory)

3 Are there established management theories that can explain the phenomenon (theory)

4 Are these antecedents or causes (theory)

5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)

6 What is the most appropriate research design to answer my research questionresearch question

Kenneth Law 同济大学 2010

When to use which design39

When to use which design

1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis

Kenneth Law 同济大学 2010

Do Not start your research40

Do Not start your research helliphellip

With some hypothesized relationships

With a model

Just because no one has done that before

Only because you see a significant correlation in your datasetyour dataset

Without looking at the literature

Without a research questionq

If you do not find it interesting

If you do not believe it is important

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

3 Laboratory designs8

3 Laboratory designs

Manipulation variables (independent variables)Manipulation variables (independent variables)Outcome variables (dependent variables)

Research questionHow would a set of outcome variables change as a result of our manipulation of some causalas a result of our manipulation of some causal variables (causality) lightning and thundering

Leadership style Employee performance

Kenneth Law 同济大学 2010

Examples9

Examples

Allen TD amp Rush MC (1998) The effects of organizational citizenship behavior on performance j d t A fi ld t d d l b t i tjudgments A field study and a laboratory experiment Journal of Applied Psychology 83(2) 247-260

Kenneth Law 同济大学 2010

10

AbAbstract

The process of linking organizational citizenship behavior (OCB) with performance judgments was investigated in a field and a laboratory study In the field study managers rated the task performance and OCB of 148 subordinates In the laboratory research 136 students viewed and ratedsubordinates In the laboratory research 136 students viewed and rated videotaped segments of teaching performance that demonstrated either high or low task performance and high or low OCB

Citizenship Behaviors Overall evaluationBehaviors

Kenneth Law 同济大学 2010

11

High OCB Low OCB

Manipulation

Results

Performance rating Performance ratingBy students (X1) gtgtgt By students (X2)

Kenneth Law 同济大学 2010

Types of experimental designs12

Types of experimental designs

1 Experimental designs

2Quasi experimentsbull Field experiments bull Time series designs

Kenneth Law 同济大学 2010

Experimental designs13

Experimental designsPretestposttest control-group designeg having one group of employees taking the training program in week 1 The second group which will get the training after group one is used as the control group

Why do we need a control group if we have pretest and posttest alreadyWhats wrong with just having posttest with control groups

XT1 T2Experimental group 1 2Experimental group

Control group T1 T2

Kenneth Law 同济大学 2010

Quasi experiment14

Quasi-experiment

- Experimental design without real manipulations- No randomization

- Hui C Lam SK amp Law KS (2000) Instrumental values of organizational citizenship behavior for promotion A field quasi-experiment Journal of Applied Psychology 85 822-828

Kenneth Law 同济大学 2010

15

Employees

P ti OCB High OCB Promotion OCB

Antecedents

Low OCB No Promotion OCB

Kenneth Law 同济大学 2010

Correlational surveys16

Correlational surveys

Predictor variables (independent variables)Criterion variables (dependent variables)

Research questionHow would a set of predictor variables affect a set of outcome

variables (causality)

Performance Promotion

time 1 time 1

Kenneth Law 同济大学 2010

Examples17

Barling J LughlinC amp Kelloway EK (2002) Development and test of a model linking safety-specific transformational leadership and occupational safety Journal of Applied Psychology 87(3) 488-496

Vancouver JB Thompson CM Tischner EC amp Putka DJ (2002) Two studies examining the negative effect of self-efficacy on performance Journal of Applied Psychology y p J f App y gy87(3) 506-516

Dvir T Eden D Avoilio BJ Shamir B (2002) ImpactDvir T Eden D Avoilio BJ Shamir B (2002) Impact of transformational leadership on follower development and performance A field experiment Academy of Management Journal 45(4) 735-744

Kenneth Law 同济大学 2010

Journal 45(4) 735-744

Survey study18

Survey study

rSafety‐specific

Occupational

rxy

transformational leadership

Occupational safety

Self‐efficacy Job performance

Transformational leadership

Job performance

Kenneth Law 同济大学 2010

Meta Analysis19

Meta Analysis

R h tiResearch questionCan we combine quantitative research in the past and summarize the research findings quantitatively

Judge T Heller D Mount M (2002) Five-Factor g ( )model of personality and job satisfaction A meta-analysis Journal of Applied Psychology 87(3) 530-541

Kenneth Law 同济大学 2010

Meta analysis20

y

Big‐5 personality individualism neuroticism conscientiousness agreeableness openness to experiences

Rxy is the correlation between agreeableness and job satisfaction

Study N rxy

1 150 35 2 247 ‐07

3 386 174 124 485 278 025 76 450 29

Kenneth Law 同济大学 2010

21

Kenneth Law 同济大学 2010

Issues in survey design 22

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Issues in survey design 23

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

1 What is your research question24

1 What is your research question Is the research question testableq

改善我国旅游业未来发展的几个意见

Are the constructs well defined 「企业进取性」和企业业绩表现的关系

Do we have validated scales to measure the constructs Existing scales Western scales

Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础

Kenneth Law 同济大学 2010

What is the research question25

qIs EQ really useful in predicting work outcomes

Would a firm using a strategic human resource management approach be more competitive

What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC

Can supervisors distinguish task performance from contextual performance in the PRC

What are the antecedents of contextual performance

Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC

Kenneth Law 同济大学 2010

生涯管理要素圖26

生涯情況 信息 計劃 資源目標

探索的動機 候迭人的信息

生活的目標

1個人戰略2時間

1 解決個人問題的技能

2 控制1自我評估價值技術興趣經驗

2組織評估表現潛力分配計劃

生涯發展目標 生涯計劃執行

分配計劃

現有內部勞資市場1工作調整需要2生涯之路結構3內部提升

生涯機會信息1生涯信息系統2生涯咨詢

1未來組織經濟目標

2未來所需職工

1組織人才資源發展戰略

2重要職務分配

1工作機會2贊助人3生涯管理者

Kenneth Law 同济大学 2010

Th di ti l f LMX

27

The mediating role of LMX

OCB

TransformationalLeadership LMX-MDM

OCB

77

3280

TaskPerformance

16

Kenneth Law 同济大学 2010

Contributions28

Contributions1 Theoretical contributions

bull New variables based on theorybull New perspectivesbull New findings

h d l l b2 Methodological contributionsbull New measures (eg new scale development)

N th d ( l l h)bull New methods (eg cross‐level research)

Kenneth Law 同济大学 2010

Identifying Research Questions29

Identifying Research Questions

员 企业的交换契约

公平理论与员工公民行为

Literature

员工企业的交换契约Incremental theoretical

contribution

Research Questions

M h d l l Practitioners公司治理

Methodological rigor

59岁现象

合资企业员工本土化

Kenneth Law 同济大学 2010

What is a research question30

What is a research question

My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust

How do you like that

Expected returnknowledge sharing

knowledge sharing

Benevolence

Expected return

Trust

s a gtendency

s a gbehaviors

Kenneth Law 同济大学 2010

A conversation hellip31

A conversation hellip

Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate

ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic

ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal

Kenneth Law 同济大学 2010

A different view32

A different view

We do research because we want to know We do research because we want to know the answer hellip

Therefore before we start a study we must have a research question

We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question

Kenneth Law 同济大学 2010

A different view33

A different view

With a research question you would automatically ask for a theoryautomatically ask for a theory

B d h ld Based on your theory you would automatically develop some hypotheses

Kenneth Law 同济大学 2010

The research process34

The research process

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Trusttendency behaviors

Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g

Kenneth Law 同济大学 2010

The research process35

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

H d k th t k l d h i i t ti i

Trust

How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g

This is because helliphellip (a theory)

Kenneth Law 同济大学 2010

The research process36

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Y d th k l d h i i t ti b

Trust

You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)

Kenneth Law 同济大学 2010

The research process37

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

(1) Some people have an in‐born tendency to share with

Trust

(1) Some people have an in‐born tendency to share with others

(2) Some people would expect reciprocal sharing in the future

(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)

(4) helliphellip

Kenneth Law 同济大学 2010

(4) helliphellip

Some questions38

1 Has anyone done anything on this topic before (literature review)

2 Which one is the most important reason to explain knowledge sharing (theory)

3 Are there established management theories that can explain the phenomenon (theory)

4 Are these antecedents or causes (theory)

5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)

6 What is the most appropriate research design to answer my research questionresearch question

Kenneth Law 同济大学 2010

When to use which design39

When to use which design

1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis

Kenneth Law 同济大学 2010

Do Not start your research40

Do Not start your research helliphellip

With some hypothesized relationships

With a model

Just because no one has done that before

Only because you see a significant correlation in your datasetyour dataset

Without looking at the literature

Without a research questionq

If you do not find it interesting

If you do not believe it is important

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Examples9

Examples

Allen TD amp Rush MC (1998) The effects of organizational citizenship behavior on performance j d t A fi ld t d d l b t i tjudgments A field study and a laboratory experiment Journal of Applied Psychology 83(2) 247-260

Kenneth Law 同济大学 2010

10

AbAbstract

The process of linking organizational citizenship behavior (OCB) with performance judgments was investigated in a field and a laboratory study In the field study managers rated the task performance and OCB of 148 subordinates In the laboratory research 136 students viewed and ratedsubordinates In the laboratory research 136 students viewed and rated videotaped segments of teaching performance that demonstrated either high or low task performance and high or low OCB

Citizenship Behaviors Overall evaluationBehaviors

Kenneth Law 同济大学 2010

11

High OCB Low OCB

Manipulation

Results

Performance rating Performance ratingBy students (X1) gtgtgt By students (X2)

Kenneth Law 同济大学 2010

Types of experimental designs12

Types of experimental designs

1 Experimental designs

2Quasi experimentsbull Field experiments bull Time series designs

Kenneth Law 同济大学 2010

Experimental designs13

Experimental designsPretestposttest control-group designeg having one group of employees taking the training program in week 1 The second group which will get the training after group one is used as the control group

Why do we need a control group if we have pretest and posttest alreadyWhats wrong with just having posttest with control groups

XT1 T2Experimental group 1 2Experimental group

Control group T1 T2

Kenneth Law 同济大学 2010

Quasi experiment14

Quasi-experiment

- Experimental design without real manipulations- No randomization

- Hui C Lam SK amp Law KS (2000) Instrumental values of organizational citizenship behavior for promotion A field quasi-experiment Journal of Applied Psychology 85 822-828

Kenneth Law 同济大学 2010

15

Employees

P ti OCB High OCB Promotion OCB

Antecedents

Low OCB No Promotion OCB

Kenneth Law 同济大学 2010

Correlational surveys16

Correlational surveys

Predictor variables (independent variables)Criterion variables (dependent variables)

Research questionHow would a set of predictor variables affect a set of outcome

variables (causality)

Performance Promotion

time 1 time 1

Kenneth Law 同济大学 2010

Examples17

Barling J LughlinC amp Kelloway EK (2002) Development and test of a model linking safety-specific transformational leadership and occupational safety Journal of Applied Psychology 87(3) 488-496

Vancouver JB Thompson CM Tischner EC amp Putka DJ (2002) Two studies examining the negative effect of self-efficacy on performance Journal of Applied Psychology y p J f App y gy87(3) 506-516

Dvir T Eden D Avoilio BJ Shamir B (2002) ImpactDvir T Eden D Avoilio BJ Shamir B (2002) Impact of transformational leadership on follower development and performance A field experiment Academy of Management Journal 45(4) 735-744

Kenneth Law 同济大学 2010

Journal 45(4) 735-744

Survey study18

Survey study

rSafety‐specific

Occupational

rxy

transformational leadership

Occupational safety

Self‐efficacy Job performance

Transformational leadership

Job performance

Kenneth Law 同济大学 2010

Meta Analysis19

Meta Analysis

R h tiResearch questionCan we combine quantitative research in the past and summarize the research findings quantitatively

Judge T Heller D Mount M (2002) Five-Factor g ( )model of personality and job satisfaction A meta-analysis Journal of Applied Psychology 87(3) 530-541

Kenneth Law 同济大学 2010

Meta analysis20

y

Big‐5 personality individualism neuroticism conscientiousness agreeableness openness to experiences

Rxy is the correlation between agreeableness and job satisfaction

Study N rxy

1 150 35 2 247 ‐07

3 386 174 124 485 278 025 76 450 29

Kenneth Law 同济大学 2010

21

Kenneth Law 同济大学 2010

Issues in survey design 22

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Issues in survey design 23

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

1 What is your research question24

1 What is your research question Is the research question testableq

改善我国旅游业未来发展的几个意见

Are the constructs well defined 「企业进取性」和企业业绩表现的关系

Do we have validated scales to measure the constructs Existing scales Western scales

Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础

Kenneth Law 同济大学 2010

What is the research question25

qIs EQ really useful in predicting work outcomes

Would a firm using a strategic human resource management approach be more competitive

What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC

Can supervisors distinguish task performance from contextual performance in the PRC

What are the antecedents of contextual performance

Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC

Kenneth Law 同济大学 2010

生涯管理要素圖26

生涯情況 信息 計劃 資源目標

探索的動機 候迭人的信息

生活的目標

1個人戰略2時間

1 解決個人問題的技能

2 控制1自我評估價值技術興趣經驗

2組織評估表現潛力分配計劃

生涯發展目標 生涯計劃執行

分配計劃

現有內部勞資市場1工作調整需要2生涯之路結構3內部提升

生涯機會信息1生涯信息系統2生涯咨詢

1未來組織經濟目標

2未來所需職工

1組織人才資源發展戰略

2重要職務分配

1工作機會2贊助人3生涯管理者

Kenneth Law 同济大学 2010

Th di ti l f LMX

27

The mediating role of LMX

OCB

TransformationalLeadership LMX-MDM

OCB

77

3280

TaskPerformance

16

Kenneth Law 同济大学 2010

Contributions28

Contributions1 Theoretical contributions

bull New variables based on theorybull New perspectivesbull New findings

h d l l b2 Methodological contributionsbull New measures (eg new scale development)

N th d ( l l h)bull New methods (eg cross‐level research)

Kenneth Law 同济大学 2010

Identifying Research Questions29

Identifying Research Questions

员 企业的交换契约

公平理论与员工公民行为

Literature

员工企业的交换契约Incremental theoretical

contribution

Research Questions

M h d l l Practitioners公司治理

Methodological rigor

59岁现象

合资企业员工本土化

Kenneth Law 同济大学 2010

What is a research question30

What is a research question

My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust

How do you like that

Expected returnknowledge sharing

knowledge sharing

Benevolence

Expected return

Trust

s a gtendency

s a gbehaviors

Kenneth Law 同济大学 2010

A conversation hellip31

A conversation hellip

Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate

ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic

ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal

Kenneth Law 同济大学 2010

A different view32

A different view

We do research because we want to know We do research because we want to know the answer hellip

Therefore before we start a study we must have a research question

We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question

Kenneth Law 同济大学 2010

A different view33

A different view

With a research question you would automatically ask for a theoryautomatically ask for a theory

B d h ld Based on your theory you would automatically develop some hypotheses

Kenneth Law 同济大学 2010

The research process34

The research process

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Trusttendency behaviors

Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g

Kenneth Law 同济大学 2010

The research process35

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

H d k th t k l d h i i t ti i

Trust

How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g

This is because helliphellip (a theory)

Kenneth Law 同济大学 2010

The research process36

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Y d th k l d h i i t ti b

Trust

You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)

Kenneth Law 同济大学 2010

The research process37

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

(1) Some people have an in‐born tendency to share with

Trust

(1) Some people have an in‐born tendency to share with others

(2) Some people would expect reciprocal sharing in the future

(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)

(4) helliphellip

Kenneth Law 同济大学 2010

(4) helliphellip

Some questions38

1 Has anyone done anything on this topic before (literature review)

2 Which one is the most important reason to explain knowledge sharing (theory)

3 Are there established management theories that can explain the phenomenon (theory)

4 Are these antecedents or causes (theory)

5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)

6 What is the most appropriate research design to answer my research questionresearch question

Kenneth Law 同济大学 2010

When to use which design39

When to use which design

1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis

Kenneth Law 同济大学 2010

Do Not start your research40

Do Not start your research helliphellip

With some hypothesized relationships

With a model

Just because no one has done that before

Only because you see a significant correlation in your datasetyour dataset

Without looking at the literature

Without a research questionq

If you do not find it interesting

If you do not believe it is important

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

10

AbAbstract

The process of linking organizational citizenship behavior (OCB) with performance judgments was investigated in a field and a laboratory study In the field study managers rated the task performance and OCB of 148 subordinates In the laboratory research 136 students viewed and ratedsubordinates In the laboratory research 136 students viewed and rated videotaped segments of teaching performance that demonstrated either high or low task performance and high or low OCB

Citizenship Behaviors Overall evaluationBehaviors

Kenneth Law 同济大学 2010

11

High OCB Low OCB

Manipulation

Results

Performance rating Performance ratingBy students (X1) gtgtgt By students (X2)

Kenneth Law 同济大学 2010

Types of experimental designs12

Types of experimental designs

1 Experimental designs

2Quasi experimentsbull Field experiments bull Time series designs

Kenneth Law 同济大学 2010

Experimental designs13

Experimental designsPretestposttest control-group designeg having one group of employees taking the training program in week 1 The second group which will get the training after group one is used as the control group

Why do we need a control group if we have pretest and posttest alreadyWhats wrong with just having posttest with control groups

XT1 T2Experimental group 1 2Experimental group

Control group T1 T2

Kenneth Law 同济大学 2010

Quasi experiment14

Quasi-experiment

- Experimental design without real manipulations- No randomization

- Hui C Lam SK amp Law KS (2000) Instrumental values of organizational citizenship behavior for promotion A field quasi-experiment Journal of Applied Psychology 85 822-828

Kenneth Law 同济大学 2010

15

Employees

P ti OCB High OCB Promotion OCB

Antecedents

Low OCB No Promotion OCB

Kenneth Law 同济大学 2010

Correlational surveys16

Correlational surveys

Predictor variables (independent variables)Criterion variables (dependent variables)

Research questionHow would a set of predictor variables affect a set of outcome

variables (causality)

Performance Promotion

time 1 time 1

Kenneth Law 同济大学 2010

Examples17

Barling J LughlinC amp Kelloway EK (2002) Development and test of a model linking safety-specific transformational leadership and occupational safety Journal of Applied Psychology 87(3) 488-496

Vancouver JB Thompson CM Tischner EC amp Putka DJ (2002) Two studies examining the negative effect of self-efficacy on performance Journal of Applied Psychology y p J f App y gy87(3) 506-516

Dvir T Eden D Avoilio BJ Shamir B (2002) ImpactDvir T Eden D Avoilio BJ Shamir B (2002) Impact of transformational leadership on follower development and performance A field experiment Academy of Management Journal 45(4) 735-744

Kenneth Law 同济大学 2010

Journal 45(4) 735-744

Survey study18

Survey study

rSafety‐specific

Occupational

rxy

transformational leadership

Occupational safety

Self‐efficacy Job performance

Transformational leadership

Job performance

Kenneth Law 同济大学 2010

Meta Analysis19

Meta Analysis

R h tiResearch questionCan we combine quantitative research in the past and summarize the research findings quantitatively

Judge T Heller D Mount M (2002) Five-Factor g ( )model of personality and job satisfaction A meta-analysis Journal of Applied Psychology 87(3) 530-541

Kenneth Law 同济大学 2010

Meta analysis20

y

Big‐5 personality individualism neuroticism conscientiousness agreeableness openness to experiences

Rxy is the correlation between agreeableness and job satisfaction

Study N rxy

1 150 35 2 247 ‐07

3 386 174 124 485 278 025 76 450 29

Kenneth Law 同济大学 2010

21

Kenneth Law 同济大学 2010

Issues in survey design 22

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Issues in survey design 23

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

1 What is your research question24

1 What is your research question Is the research question testableq

改善我国旅游业未来发展的几个意见

Are the constructs well defined 「企业进取性」和企业业绩表现的关系

Do we have validated scales to measure the constructs Existing scales Western scales

Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础

Kenneth Law 同济大学 2010

What is the research question25

qIs EQ really useful in predicting work outcomes

Would a firm using a strategic human resource management approach be more competitive

What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC

Can supervisors distinguish task performance from contextual performance in the PRC

What are the antecedents of contextual performance

Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC

Kenneth Law 同济大学 2010

生涯管理要素圖26

生涯情況 信息 計劃 資源目標

探索的動機 候迭人的信息

生活的目標

1個人戰略2時間

1 解決個人問題的技能

2 控制1自我評估價值技術興趣經驗

2組織評估表現潛力分配計劃

生涯發展目標 生涯計劃執行

分配計劃

現有內部勞資市場1工作調整需要2生涯之路結構3內部提升

生涯機會信息1生涯信息系統2生涯咨詢

1未來組織經濟目標

2未來所需職工

1組織人才資源發展戰略

2重要職務分配

1工作機會2贊助人3生涯管理者

Kenneth Law 同济大学 2010

Th di ti l f LMX

27

The mediating role of LMX

OCB

TransformationalLeadership LMX-MDM

OCB

77

3280

TaskPerformance

16

Kenneth Law 同济大学 2010

Contributions28

Contributions1 Theoretical contributions

bull New variables based on theorybull New perspectivesbull New findings

h d l l b2 Methodological contributionsbull New measures (eg new scale development)

N th d ( l l h)bull New methods (eg cross‐level research)

Kenneth Law 同济大学 2010

Identifying Research Questions29

Identifying Research Questions

员 企业的交换契约

公平理论与员工公民行为

Literature

员工企业的交换契约Incremental theoretical

contribution

Research Questions

M h d l l Practitioners公司治理

Methodological rigor

59岁现象

合资企业员工本土化

Kenneth Law 同济大学 2010

What is a research question30

What is a research question

My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust

How do you like that

Expected returnknowledge sharing

knowledge sharing

Benevolence

Expected return

Trust

s a gtendency

s a gbehaviors

Kenneth Law 同济大学 2010

A conversation hellip31

A conversation hellip

Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate

ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic

ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal

Kenneth Law 同济大学 2010

A different view32

A different view

We do research because we want to know We do research because we want to know the answer hellip

Therefore before we start a study we must have a research question

We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question

Kenneth Law 同济大学 2010

A different view33

A different view

With a research question you would automatically ask for a theoryautomatically ask for a theory

B d h ld Based on your theory you would automatically develop some hypotheses

Kenneth Law 同济大学 2010

The research process34

The research process

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Trusttendency behaviors

Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g

Kenneth Law 同济大学 2010

The research process35

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

H d k th t k l d h i i t ti i

Trust

How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g

This is because helliphellip (a theory)

Kenneth Law 同济大学 2010

The research process36

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Y d th k l d h i i t ti b

Trust

You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)

Kenneth Law 同济大学 2010

The research process37

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

(1) Some people have an in‐born tendency to share with

Trust

(1) Some people have an in‐born tendency to share with others

(2) Some people would expect reciprocal sharing in the future

(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)

(4) helliphellip

Kenneth Law 同济大学 2010

(4) helliphellip

Some questions38

1 Has anyone done anything on this topic before (literature review)

2 Which one is the most important reason to explain knowledge sharing (theory)

3 Are there established management theories that can explain the phenomenon (theory)

4 Are these antecedents or causes (theory)

5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)

6 What is the most appropriate research design to answer my research questionresearch question

Kenneth Law 同济大学 2010

When to use which design39

When to use which design

1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis

Kenneth Law 同济大学 2010

Do Not start your research40

Do Not start your research helliphellip

With some hypothesized relationships

With a model

Just because no one has done that before

Only because you see a significant correlation in your datasetyour dataset

Without looking at the literature

Without a research questionq

If you do not find it interesting

If you do not believe it is important

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

11

High OCB Low OCB

Manipulation

Results

Performance rating Performance ratingBy students (X1) gtgtgt By students (X2)

Kenneth Law 同济大学 2010

Types of experimental designs12

Types of experimental designs

1 Experimental designs

2Quasi experimentsbull Field experiments bull Time series designs

Kenneth Law 同济大学 2010

Experimental designs13

Experimental designsPretestposttest control-group designeg having one group of employees taking the training program in week 1 The second group which will get the training after group one is used as the control group

Why do we need a control group if we have pretest and posttest alreadyWhats wrong with just having posttest with control groups

XT1 T2Experimental group 1 2Experimental group

Control group T1 T2

Kenneth Law 同济大学 2010

Quasi experiment14

Quasi-experiment

- Experimental design without real manipulations- No randomization

- Hui C Lam SK amp Law KS (2000) Instrumental values of organizational citizenship behavior for promotion A field quasi-experiment Journal of Applied Psychology 85 822-828

Kenneth Law 同济大学 2010

15

Employees

P ti OCB High OCB Promotion OCB

Antecedents

Low OCB No Promotion OCB

Kenneth Law 同济大学 2010

Correlational surveys16

Correlational surveys

Predictor variables (independent variables)Criterion variables (dependent variables)

Research questionHow would a set of predictor variables affect a set of outcome

variables (causality)

Performance Promotion

time 1 time 1

Kenneth Law 同济大学 2010

Examples17

Barling J LughlinC amp Kelloway EK (2002) Development and test of a model linking safety-specific transformational leadership and occupational safety Journal of Applied Psychology 87(3) 488-496

Vancouver JB Thompson CM Tischner EC amp Putka DJ (2002) Two studies examining the negative effect of self-efficacy on performance Journal of Applied Psychology y p J f App y gy87(3) 506-516

Dvir T Eden D Avoilio BJ Shamir B (2002) ImpactDvir T Eden D Avoilio BJ Shamir B (2002) Impact of transformational leadership on follower development and performance A field experiment Academy of Management Journal 45(4) 735-744

Kenneth Law 同济大学 2010

Journal 45(4) 735-744

Survey study18

Survey study

rSafety‐specific

Occupational

rxy

transformational leadership

Occupational safety

Self‐efficacy Job performance

Transformational leadership

Job performance

Kenneth Law 同济大学 2010

Meta Analysis19

Meta Analysis

R h tiResearch questionCan we combine quantitative research in the past and summarize the research findings quantitatively

Judge T Heller D Mount M (2002) Five-Factor g ( )model of personality and job satisfaction A meta-analysis Journal of Applied Psychology 87(3) 530-541

Kenneth Law 同济大学 2010

Meta analysis20

y

Big‐5 personality individualism neuroticism conscientiousness agreeableness openness to experiences

Rxy is the correlation between agreeableness and job satisfaction

Study N rxy

1 150 35 2 247 ‐07

3 386 174 124 485 278 025 76 450 29

Kenneth Law 同济大学 2010

21

Kenneth Law 同济大学 2010

Issues in survey design 22

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Issues in survey design 23

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

1 What is your research question24

1 What is your research question Is the research question testableq

改善我国旅游业未来发展的几个意见

Are the constructs well defined 「企业进取性」和企业业绩表现的关系

Do we have validated scales to measure the constructs Existing scales Western scales

Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础

Kenneth Law 同济大学 2010

What is the research question25

qIs EQ really useful in predicting work outcomes

Would a firm using a strategic human resource management approach be more competitive

What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC

Can supervisors distinguish task performance from contextual performance in the PRC

What are the antecedents of contextual performance

Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC

Kenneth Law 同济大学 2010

生涯管理要素圖26

生涯情況 信息 計劃 資源目標

探索的動機 候迭人的信息

生活的目標

1個人戰略2時間

1 解決個人問題的技能

2 控制1自我評估價值技術興趣經驗

2組織評估表現潛力分配計劃

生涯發展目標 生涯計劃執行

分配計劃

現有內部勞資市場1工作調整需要2生涯之路結構3內部提升

生涯機會信息1生涯信息系統2生涯咨詢

1未來組織經濟目標

2未來所需職工

1組織人才資源發展戰略

2重要職務分配

1工作機會2贊助人3生涯管理者

Kenneth Law 同济大学 2010

Th di ti l f LMX

27

The mediating role of LMX

OCB

TransformationalLeadership LMX-MDM

OCB

77

3280

TaskPerformance

16

Kenneth Law 同济大学 2010

Contributions28

Contributions1 Theoretical contributions

bull New variables based on theorybull New perspectivesbull New findings

h d l l b2 Methodological contributionsbull New measures (eg new scale development)

N th d ( l l h)bull New methods (eg cross‐level research)

Kenneth Law 同济大学 2010

Identifying Research Questions29

Identifying Research Questions

员 企业的交换契约

公平理论与员工公民行为

Literature

员工企业的交换契约Incremental theoretical

contribution

Research Questions

M h d l l Practitioners公司治理

Methodological rigor

59岁现象

合资企业员工本土化

Kenneth Law 同济大学 2010

What is a research question30

What is a research question

My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust

How do you like that

Expected returnknowledge sharing

knowledge sharing

Benevolence

Expected return

Trust

s a gtendency

s a gbehaviors

Kenneth Law 同济大学 2010

A conversation hellip31

A conversation hellip

Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate

ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic

ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal

Kenneth Law 同济大学 2010

A different view32

A different view

We do research because we want to know We do research because we want to know the answer hellip

Therefore before we start a study we must have a research question

We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question

Kenneth Law 同济大学 2010

A different view33

A different view

With a research question you would automatically ask for a theoryautomatically ask for a theory

B d h ld Based on your theory you would automatically develop some hypotheses

Kenneth Law 同济大学 2010

The research process34

The research process

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Trusttendency behaviors

Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g

Kenneth Law 同济大学 2010

The research process35

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

H d k th t k l d h i i t ti i

Trust

How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g

This is because helliphellip (a theory)

Kenneth Law 同济大学 2010

The research process36

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Y d th k l d h i i t ti b

Trust

You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)

Kenneth Law 同济大学 2010

The research process37

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

(1) Some people have an in‐born tendency to share with

Trust

(1) Some people have an in‐born tendency to share with others

(2) Some people would expect reciprocal sharing in the future

(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)

(4) helliphellip

Kenneth Law 同济大学 2010

(4) helliphellip

Some questions38

1 Has anyone done anything on this topic before (literature review)

2 Which one is the most important reason to explain knowledge sharing (theory)

3 Are there established management theories that can explain the phenomenon (theory)

4 Are these antecedents or causes (theory)

5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)

6 What is the most appropriate research design to answer my research questionresearch question

Kenneth Law 同济大学 2010

When to use which design39

When to use which design

1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis

Kenneth Law 同济大学 2010

Do Not start your research40

Do Not start your research helliphellip

With some hypothesized relationships

With a model

Just because no one has done that before

Only because you see a significant correlation in your datasetyour dataset

Without looking at the literature

Without a research questionq

If you do not find it interesting

If you do not believe it is important

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Types of experimental designs12

Types of experimental designs

1 Experimental designs

2Quasi experimentsbull Field experiments bull Time series designs

Kenneth Law 同济大学 2010

Experimental designs13

Experimental designsPretestposttest control-group designeg having one group of employees taking the training program in week 1 The second group which will get the training after group one is used as the control group

Why do we need a control group if we have pretest and posttest alreadyWhats wrong with just having posttest with control groups

XT1 T2Experimental group 1 2Experimental group

Control group T1 T2

Kenneth Law 同济大学 2010

Quasi experiment14

Quasi-experiment

- Experimental design without real manipulations- No randomization

- Hui C Lam SK amp Law KS (2000) Instrumental values of organizational citizenship behavior for promotion A field quasi-experiment Journal of Applied Psychology 85 822-828

Kenneth Law 同济大学 2010

15

Employees

P ti OCB High OCB Promotion OCB

Antecedents

Low OCB No Promotion OCB

Kenneth Law 同济大学 2010

Correlational surveys16

Correlational surveys

Predictor variables (independent variables)Criterion variables (dependent variables)

Research questionHow would a set of predictor variables affect a set of outcome

variables (causality)

Performance Promotion

time 1 time 1

Kenneth Law 同济大学 2010

Examples17

Barling J LughlinC amp Kelloway EK (2002) Development and test of a model linking safety-specific transformational leadership and occupational safety Journal of Applied Psychology 87(3) 488-496

Vancouver JB Thompson CM Tischner EC amp Putka DJ (2002) Two studies examining the negative effect of self-efficacy on performance Journal of Applied Psychology y p J f App y gy87(3) 506-516

Dvir T Eden D Avoilio BJ Shamir B (2002) ImpactDvir T Eden D Avoilio BJ Shamir B (2002) Impact of transformational leadership on follower development and performance A field experiment Academy of Management Journal 45(4) 735-744

Kenneth Law 同济大学 2010

Journal 45(4) 735-744

Survey study18

Survey study

rSafety‐specific

Occupational

rxy

transformational leadership

Occupational safety

Self‐efficacy Job performance

Transformational leadership

Job performance

Kenneth Law 同济大学 2010

Meta Analysis19

Meta Analysis

R h tiResearch questionCan we combine quantitative research in the past and summarize the research findings quantitatively

Judge T Heller D Mount M (2002) Five-Factor g ( )model of personality and job satisfaction A meta-analysis Journal of Applied Psychology 87(3) 530-541

Kenneth Law 同济大学 2010

Meta analysis20

y

Big‐5 personality individualism neuroticism conscientiousness agreeableness openness to experiences

Rxy is the correlation between agreeableness and job satisfaction

Study N rxy

1 150 35 2 247 ‐07

3 386 174 124 485 278 025 76 450 29

Kenneth Law 同济大学 2010

21

Kenneth Law 同济大学 2010

Issues in survey design 22

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Issues in survey design 23

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

1 What is your research question24

1 What is your research question Is the research question testableq

改善我国旅游业未来发展的几个意见

Are the constructs well defined 「企业进取性」和企业业绩表现的关系

Do we have validated scales to measure the constructs Existing scales Western scales

Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础

Kenneth Law 同济大学 2010

What is the research question25

qIs EQ really useful in predicting work outcomes

Would a firm using a strategic human resource management approach be more competitive

What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC

Can supervisors distinguish task performance from contextual performance in the PRC

What are the antecedents of contextual performance

Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC

Kenneth Law 同济大学 2010

生涯管理要素圖26

生涯情況 信息 計劃 資源目標

探索的動機 候迭人的信息

生活的目標

1個人戰略2時間

1 解決個人問題的技能

2 控制1自我評估價值技術興趣經驗

2組織評估表現潛力分配計劃

生涯發展目標 生涯計劃執行

分配計劃

現有內部勞資市場1工作調整需要2生涯之路結構3內部提升

生涯機會信息1生涯信息系統2生涯咨詢

1未來組織經濟目標

2未來所需職工

1組織人才資源發展戰略

2重要職務分配

1工作機會2贊助人3生涯管理者

Kenneth Law 同济大学 2010

Th di ti l f LMX

27

The mediating role of LMX

OCB

TransformationalLeadership LMX-MDM

OCB

77

3280

TaskPerformance

16

Kenneth Law 同济大学 2010

Contributions28

Contributions1 Theoretical contributions

bull New variables based on theorybull New perspectivesbull New findings

h d l l b2 Methodological contributionsbull New measures (eg new scale development)

N th d ( l l h)bull New methods (eg cross‐level research)

Kenneth Law 同济大学 2010

Identifying Research Questions29

Identifying Research Questions

员 企业的交换契约

公平理论与员工公民行为

Literature

员工企业的交换契约Incremental theoretical

contribution

Research Questions

M h d l l Practitioners公司治理

Methodological rigor

59岁现象

合资企业员工本土化

Kenneth Law 同济大学 2010

What is a research question30

What is a research question

My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust

How do you like that

Expected returnknowledge sharing

knowledge sharing

Benevolence

Expected return

Trust

s a gtendency

s a gbehaviors

Kenneth Law 同济大学 2010

A conversation hellip31

A conversation hellip

Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate

ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic

ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal

Kenneth Law 同济大学 2010

A different view32

A different view

We do research because we want to know We do research because we want to know the answer hellip

Therefore before we start a study we must have a research question

We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question

Kenneth Law 同济大学 2010

A different view33

A different view

With a research question you would automatically ask for a theoryautomatically ask for a theory

B d h ld Based on your theory you would automatically develop some hypotheses

Kenneth Law 同济大学 2010

The research process34

The research process

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Trusttendency behaviors

Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g

Kenneth Law 同济大学 2010

The research process35

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

H d k th t k l d h i i t ti i

Trust

How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g

This is because helliphellip (a theory)

Kenneth Law 同济大学 2010

The research process36

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Y d th k l d h i i t ti b

Trust

You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)

Kenneth Law 同济大学 2010

The research process37

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

(1) Some people have an in‐born tendency to share with

Trust

(1) Some people have an in‐born tendency to share with others

(2) Some people would expect reciprocal sharing in the future

(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)

(4) helliphellip

Kenneth Law 同济大学 2010

(4) helliphellip

Some questions38

1 Has anyone done anything on this topic before (literature review)

2 Which one is the most important reason to explain knowledge sharing (theory)

3 Are there established management theories that can explain the phenomenon (theory)

4 Are these antecedents or causes (theory)

5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)

6 What is the most appropriate research design to answer my research questionresearch question

Kenneth Law 同济大学 2010

When to use which design39

When to use which design

1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis

Kenneth Law 同济大学 2010

Do Not start your research40

Do Not start your research helliphellip

With some hypothesized relationships

With a model

Just because no one has done that before

Only because you see a significant correlation in your datasetyour dataset

Without looking at the literature

Without a research questionq

If you do not find it interesting

If you do not believe it is important

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Experimental designs13

Experimental designsPretestposttest control-group designeg having one group of employees taking the training program in week 1 The second group which will get the training after group one is used as the control group

Why do we need a control group if we have pretest and posttest alreadyWhats wrong with just having posttest with control groups

XT1 T2Experimental group 1 2Experimental group

Control group T1 T2

Kenneth Law 同济大学 2010

Quasi experiment14

Quasi-experiment

- Experimental design without real manipulations- No randomization

- Hui C Lam SK amp Law KS (2000) Instrumental values of organizational citizenship behavior for promotion A field quasi-experiment Journal of Applied Psychology 85 822-828

Kenneth Law 同济大学 2010

15

Employees

P ti OCB High OCB Promotion OCB

Antecedents

Low OCB No Promotion OCB

Kenneth Law 同济大学 2010

Correlational surveys16

Correlational surveys

Predictor variables (independent variables)Criterion variables (dependent variables)

Research questionHow would a set of predictor variables affect a set of outcome

variables (causality)

Performance Promotion

time 1 time 1

Kenneth Law 同济大学 2010

Examples17

Barling J LughlinC amp Kelloway EK (2002) Development and test of a model linking safety-specific transformational leadership and occupational safety Journal of Applied Psychology 87(3) 488-496

Vancouver JB Thompson CM Tischner EC amp Putka DJ (2002) Two studies examining the negative effect of self-efficacy on performance Journal of Applied Psychology y p J f App y gy87(3) 506-516

Dvir T Eden D Avoilio BJ Shamir B (2002) ImpactDvir T Eden D Avoilio BJ Shamir B (2002) Impact of transformational leadership on follower development and performance A field experiment Academy of Management Journal 45(4) 735-744

Kenneth Law 同济大学 2010

Journal 45(4) 735-744

Survey study18

Survey study

rSafety‐specific

Occupational

rxy

transformational leadership

Occupational safety

Self‐efficacy Job performance

Transformational leadership

Job performance

Kenneth Law 同济大学 2010

Meta Analysis19

Meta Analysis

R h tiResearch questionCan we combine quantitative research in the past and summarize the research findings quantitatively

Judge T Heller D Mount M (2002) Five-Factor g ( )model of personality and job satisfaction A meta-analysis Journal of Applied Psychology 87(3) 530-541

Kenneth Law 同济大学 2010

Meta analysis20

y

Big‐5 personality individualism neuroticism conscientiousness agreeableness openness to experiences

Rxy is the correlation between agreeableness and job satisfaction

Study N rxy

1 150 35 2 247 ‐07

3 386 174 124 485 278 025 76 450 29

Kenneth Law 同济大学 2010

21

Kenneth Law 同济大学 2010

Issues in survey design 22

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Issues in survey design 23

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

1 What is your research question24

1 What is your research question Is the research question testableq

改善我国旅游业未来发展的几个意见

Are the constructs well defined 「企业进取性」和企业业绩表现的关系

Do we have validated scales to measure the constructs Existing scales Western scales

Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础

Kenneth Law 同济大学 2010

What is the research question25

qIs EQ really useful in predicting work outcomes

Would a firm using a strategic human resource management approach be more competitive

What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC

Can supervisors distinguish task performance from contextual performance in the PRC

What are the antecedents of contextual performance

Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC

Kenneth Law 同济大学 2010

生涯管理要素圖26

生涯情況 信息 計劃 資源目標

探索的動機 候迭人的信息

生活的目標

1個人戰略2時間

1 解決個人問題的技能

2 控制1自我評估價值技術興趣經驗

2組織評估表現潛力分配計劃

生涯發展目標 生涯計劃執行

分配計劃

現有內部勞資市場1工作調整需要2生涯之路結構3內部提升

生涯機會信息1生涯信息系統2生涯咨詢

1未來組織經濟目標

2未來所需職工

1組織人才資源發展戰略

2重要職務分配

1工作機會2贊助人3生涯管理者

Kenneth Law 同济大学 2010

Th di ti l f LMX

27

The mediating role of LMX

OCB

TransformationalLeadership LMX-MDM

OCB

77

3280

TaskPerformance

16

Kenneth Law 同济大学 2010

Contributions28

Contributions1 Theoretical contributions

bull New variables based on theorybull New perspectivesbull New findings

h d l l b2 Methodological contributionsbull New measures (eg new scale development)

N th d ( l l h)bull New methods (eg cross‐level research)

Kenneth Law 同济大学 2010

Identifying Research Questions29

Identifying Research Questions

员 企业的交换契约

公平理论与员工公民行为

Literature

员工企业的交换契约Incremental theoretical

contribution

Research Questions

M h d l l Practitioners公司治理

Methodological rigor

59岁现象

合资企业员工本土化

Kenneth Law 同济大学 2010

What is a research question30

What is a research question

My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust

How do you like that

Expected returnknowledge sharing

knowledge sharing

Benevolence

Expected return

Trust

s a gtendency

s a gbehaviors

Kenneth Law 同济大学 2010

A conversation hellip31

A conversation hellip

Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate

ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic

ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal

Kenneth Law 同济大学 2010

A different view32

A different view

We do research because we want to know We do research because we want to know the answer hellip

Therefore before we start a study we must have a research question

We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question

Kenneth Law 同济大学 2010

A different view33

A different view

With a research question you would automatically ask for a theoryautomatically ask for a theory

B d h ld Based on your theory you would automatically develop some hypotheses

Kenneth Law 同济大学 2010

The research process34

The research process

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Trusttendency behaviors

Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g

Kenneth Law 同济大学 2010

The research process35

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

H d k th t k l d h i i t ti i

Trust

How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g

This is because helliphellip (a theory)

Kenneth Law 同济大学 2010

The research process36

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Y d th k l d h i i t ti b

Trust

You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)

Kenneth Law 同济大学 2010

The research process37

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

(1) Some people have an in‐born tendency to share with

Trust

(1) Some people have an in‐born tendency to share with others

(2) Some people would expect reciprocal sharing in the future

(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)

(4) helliphellip

Kenneth Law 同济大学 2010

(4) helliphellip

Some questions38

1 Has anyone done anything on this topic before (literature review)

2 Which one is the most important reason to explain knowledge sharing (theory)

3 Are there established management theories that can explain the phenomenon (theory)

4 Are these antecedents or causes (theory)

5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)

6 What is the most appropriate research design to answer my research questionresearch question

Kenneth Law 同济大学 2010

When to use which design39

When to use which design

1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis

Kenneth Law 同济大学 2010

Do Not start your research40

Do Not start your research helliphellip

With some hypothesized relationships

With a model

Just because no one has done that before

Only because you see a significant correlation in your datasetyour dataset

Without looking at the literature

Without a research questionq

If you do not find it interesting

If you do not believe it is important

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Quasi experiment14

Quasi-experiment

- Experimental design without real manipulations- No randomization

- Hui C Lam SK amp Law KS (2000) Instrumental values of organizational citizenship behavior for promotion A field quasi-experiment Journal of Applied Psychology 85 822-828

Kenneth Law 同济大学 2010

15

Employees

P ti OCB High OCB Promotion OCB

Antecedents

Low OCB No Promotion OCB

Kenneth Law 同济大学 2010

Correlational surveys16

Correlational surveys

Predictor variables (independent variables)Criterion variables (dependent variables)

Research questionHow would a set of predictor variables affect a set of outcome

variables (causality)

Performance Promotion

time 1 time 1

Kenneth Law 同济大学 2010

Examples17

Barling J LughlinC amp Kelloway EK (2002) Development and test of a model linking safety-specific transformational leadership and occupational safety Journal of Applied Psychology 87(3) 488-496

Vancouver JB Thompson CM Tischner EC amp Putka DJ (2002) Two studies examining the negative effect of self-efficacy on performance Journal of Applied Psychology y p J f App y gy87(3) 506-516

Dvir T Eden D Avoilio BJ Shamir B (2002) ImpactDvir T Eden D Avoilio BJ Shamir B (2002) Impact of transformational leadership on follower development and performance A field experiment Academy of Management Journal 45(4) 735-744

Kenneth Law 同济大学 2010

Journal 45(4) 735-744

Survey study18

Survey study

rSafety‐specific

Occupational

rxy

transformational leadership

Occupational safety

Self‐efficacy Job performance

Transformational leadership

Job performance

Kenneth Law 同济大学 2010

Meta Analysis19

Meta Analysis

R h tiResearch questionCan we combine quantitative research in the past and summarize the research findings quantitatively

Judge T Heller D Mount M (2002) Five-Factor g ( )model of personality and job satisfaction A meta-analysis Journal of Applied Psychology 87(3) 530-541

Kenneth Law 同济大学 2010

Meta analysis20

y

Big‐5 personality individualism neuroticism conscientiousness agreeableness openness to experiences

Rxy is the correlation between agreeableness and job satisfaction

Study N rxy

1 150 35 2 247 ‐07

3 386 174 124 485 278 025 76 450 29

Kenneth Law 同济大学 2010

21

Kenneth Law 同济大学 2010

Issues in survey design 22

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Issues in survey design 23

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

1 What is your research question24

1 What is your research question Is the research question testableq

改善我国旅游业未来发展的几个意见

Are the constructs well defined 「企业进取性」和企业业绩表现的关系

Do we have validated scales to measure the constructs Existing scales Western scales

Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础

Kenneth Law 同济大学 2010

What is the research question25

qIs EQ really useful in predicting work outcomes

Would a firm using a strategic human resource management approach be more competitive

What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC

Can supervisors distinguish task performance from contextual performance in the PRC

What are the antecedents of contextual performance

Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC

Kenneth Law 同济大学 2010

生涯管理要素圖26

生涯情況 信息 計劃 資源目標

探索的動機 候迭人的信息

生活的目標

1個人戰略2時間

1 解決個人問題的技能

2 控制1自我評估價值技術興趣經驗

2組織評估表現潛力分配計劃

生涯發展目標 生涯計劃執行

分配計劃

現有內部勞資市場1工作調整需要2生涯之路結構3內部提升

生涯機會信息1生涯信息系統2生涯咨詢

1未來組織經濟目標

2未來所需職工

1組織人才資源發展戰略

2重要職務分配

1工作機會2贊助人3生涯管理者

Kenneth Law 同济大学 2010

Th di ti l f LMX

27

The mediating role of LMX

OCB

TransformationalLeadership LMX-MDM

OCB

77

3280

TaskPerformance

16

Kenneth Law 同济大学 2010

Contributions28

Contributions1 Theoretical contributions

bull New variables based on theorybull New perspectivesbull New findings

h d l l b2 Methodological contributionsbull New measures (eg new scale development)

N th d ( l l h)bull New methods (eg cross‐level research)

Kenneth Law 同济大学 2010

Identifying Research Questions29

Identifying Research Questions

员 企业的交换契约

公平理论与员工公民行为

Literature

员工企业的交换契约Incremental theoretical

contribution

Research Questions

M h d l l Practitioners公司治理

Methodological rigor

59岁现象

合资企业员工本土化

Kenneth Law 同济大学 2010

What is a research question30

What is a research question

My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust

How do you like that

Expected returnknowledge sharing

knowledge sharing

Benevolence

Expected return

Trust

s a gtendency

s a gbehaviors

Kenneth Law 同济大学 2010

A conversation hellip31

A conversation hellip

Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate

ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic

ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal

Kenneth Law 同济大学 2010

A different view32

A different view

We do research because we want to know We do research because we want to know the answer hellip

Therefore before we start a study we must have a research question

We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question

Kenneth Law 同济大学 2010

A different view33

A different view

With a research question you would automatically ask for a theoryautomatically ask for a theory

B d h ld Based on your theory you would automatically develop some hypotheses

Kenneth Law 同济大学 2010

The research process34

The research process

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Trusttendency behaviors

Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g

Kenneth Law 同济大学 2010

The research process35

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

H d k th t k l d h i i t ti i

Trust

How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g

This is because helliphellip (a theory)

Kenneth Law 同济大学 2010

The research process36

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Y d th k l d h i i t ti b

Trust

You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)

Kenneth Law 同济大学 2010

The research process37

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

(1) Some people have an in‐born tendency to share with

Trust

(1) Some people have an in‐born tendency to share with others

(2) Some people would expect reciprocal sharing in the future

(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)

(4) helliphellip

Kenneth Law 同济大学 2010

(4) helliphellip

Some questions38

1 Has anyone done anything on this topic before (literature review)

2 Which one is the most important reason to explain knowledge sharing (theory)

3 Are there established management theories that can explain the phenomenon (theory)

4 Are these antecedents or causes (theory)

5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)

6 What is the most appropriate research design to answer my research questionresearch question

Kenneth Law 同济大学 2010

When to use which design39

When to use which design

1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis

Kenneth Law 同济大学 2010

Do Not start your research40

Do Not start your research helliphellip

With some hypothesized relationships

With a model

Just because no one has done that before

Only because you see a significant correlation in your datasetyour dataset

Without looking at the literature

Without a research questionq

If you do not find it interesting

If you do not believe it is important

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

15

Employees

P ti OCB High OCB Promotion OCB

Antecedents

Low OCB No Promotion OCB

Kenneth Law 同济大学 2010

Correlational surveys16

Correlational surveys

Predictor variables (independent variables)Criterion variables (dependent variables)

Research questionHow would a set of predictor variables affect a set of outcome

variables (causality)

Performance Promotion

time 1 time 1

Kenneth Law 同济大学 2010

Examples17

Barling J LughlinC amp Kelloway EK (2002) Development and test of a model linking safety-specific transformational leadership and occupational safety Journal of Applied Psychology 87(3) 488-496

Vancouver JB Thompson CM Tischner EC amp Putka DJ (2002) Two studies examining the negative effect of self-efficacy on performance Journal of Applied Psychology y p J f App y gy87(3) 506-516

Dvir T Eden D Avoilio BJ Shamir B (2002) ImpactDvir T Eden D Avoilio BJ Shamir B (2002) Impact of transformational leadership on follower development and performance A field experiment Academy of Management Journal 45(4) 735-744

Kenneth Law 同济大学 2010

Journal 45(4) 735-744

Survey study18

Survey study

rSafety‐specific

Occupational

rxy

transformational leadership

Occupational safety

Self‐efficacy Job performance

Transformational leadership

Job performance

Kenneth Law 同济大学 2010

Meta Analysis19

Meta Analysis

R h tiResearch questionCan we combine quantitative research in the past and summarize the research findings quantitatively

Judge T Heller D Mount M (2002) Five-Factor g ( )model of personality and job satisfaction A meta-analysis Journal of Applied Psychology 87(3) 530-541

Kenneth Law 同济大学 2010

Meta analysis20

y

Big‐5 personality individualism neuroticism conscientiousness agreeableness openness to experiences

Rxy is the correlation between agreeableness and job satisfaction

Study N rxy

1 150 35 2 247 ‐07

3 386 174 124 485 278 025 76 450 29

Kenneth Law 同济大学 2010

21

Kenneth Law 同济大学 2010

Issues in survey design 22

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Issues in survey design 23

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

1 What is your research question24

1 What is your research question Is the research question testableq

改善我国旅游业未来发展的几个意见

Are the constructs well defined 「企业进取性」和企业业绩表现的关系

Do we have validated scales to measure the constructs Existing scales Western scales

Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础

Kenneth Law 同济大学 2010

What is the research question25

qIs EQ really useful in predicting work outcomes

Would a firm using a strategic human resource management approach be more competitive

What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC

Can supervisors distinguish task performance from contextual performance in the PRC

What are the antecedents of contextual performance

Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC

Kenneth Law 同济大学 2010

生涯管理要素圖26

生涯情況 信息 計劃 資源目標

探索的動機 候迭人的信息

生活的目標

1個人戰略2時間

1 解決個人問題的技能

2 控制1自我評估價值技術興趣經驗

2組織評估表現潛力分配計劃

生涯發展目標 生涯計劃執行

分配計劃

現有內部勞資市場1工作調整需要2生涯之路結構3內部提升

生涯機會信息1生涯信息系統2生涯咨詢

1未來組織經濟目標

2未來所需職工

1組織人才資源發展戰略

2重要職務分配

1工作機會2贊助人3生涯管理者

Kenneth Law 同济大学 2010

Th di ti l f LMX

27

The mediating role of LMX

OCB

TransformationalLeadership LMX-MDM

OCB

77

3280

TaskPerformance

16

Kenneth Law 同济大学 2010

Contributions28

Contributions1 Theoretical contributions

bull New variables based on theorybull New perspectivesbull New findings

h d l l b2 Methodological contributionsbull New measures (eg new scale development)

N th d ( l l h)bull New methods (eg cross‐level research)

Kenneth Law 同济大学 2010

Identifying Research Questions29

Identifying Research Questions

员 企业的交换契约

公平理论与员工公民行为

Literature

员工企业的交换契约Incremental theoretical

contribution

Research Questions

M h d l l Practitioners公司治理

Methodological rigor

59岁现象

合资企业员工本土化

Kenneth Law 同济大学 2010

What is a research question30

What is a research question

My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust

How do you like that

Expected returnknowledge sharing

knowledge sharing

Benevolence

Expected return

Trust

s a gtendency

s a gbehaviors

Kenneth Law 同济大学 2010

A conversation hellip31

A conversation hellip

Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate

ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic

ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal

Kenneth Law 同济大学 2010

A different view32

A different view

We do research because we want to know We do research because we want to know the answer hellip

Therefore before we start a study we must have a research question

We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question

Kenneth Law 同济大学 2010

A different view33

A different view

With a research question you would automatically ask for a theoryautomatically ask for a theory

B d h ld Based on your theory you would automatically develop some hypotheses

Kenneth Law 同济大学 2010

The research process34

The research process

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Trusttendency behaviors

Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g

Kenneth Law 同济大学 2010

The research process35

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

H d k th t k l d h i i t ti i

Trust

How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g

This is because helliphellip (a theory)

Kenneth Law 同济大学 2010

The research process36

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Y d th k l d h i i t ti b

Trust

You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)

Kenneth Law 同济大学 2010

The research process37

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

(1) Some people have an in‐born tendency to share with

Trust

(1) Some people have an in‐born tendency to share with others

(2) Some people would expect reciprocal sharing in the future

(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)

(4) helliphellip

Kenneth Law 同济大学 2010

(4) helliphellip

Some questions38

1 Has anyone done anything on this topic before (literature review)

2 Which one is the most important reason to explain knowledge sharing (theory)

3 Are there established management theories that can explain the phenomenon (theory)

4 Are these antecedents or causes (theory)

5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)

6 What is the most appropriate research design to answer my research questionresearch question

Kenneth Law 同济大学 2010

When to use which design39

When to use which design

1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis

Kenneth Law 同济大学 2010

Do Not start your research40

Do Not start your research helliphellip

With some hypothesized relationships

With a model

Just because no one has done that before

Only because you see a significant correlation in your datasetyour dataset

Without looking at the literature

Without a research questionq

If you do not find it interesting

If you do not believe it is important

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Correlational surveys16

Correlational surveys

Predictor variables (independent variables)Criterion variables (dependent variables)

Research questionHow would a set of predictor variables affect a set of outcome

variables (causality)

Performance Promotion

time 1 time 1

Kenneth Law 同济大学 2010

Examples17

Barling J LughlinC amp Kelloway EK (2002) Development and test of a model linking safety-specific transformational leadership and occupational safety Journal of Applied Psychology 87(3) 488-496

Vancouver JB Thompson CM Tischner EC amp Putka DJ (2002) Two studies examining the negative effect of self-efficacy on performance Journal of Applied Psychology y p J f App y gy87(3) 506-516

Dvir T Eden D Avoilio BJ Shamir B (2002) ImpactDvir T Eden D Avoilio BJ Shamir B (2002) Impact of transformational leadership on follower development and performance A field experiment Academy of Management Journal 45(4) 735-744

Kenneth Law 同济大学 2010

Journal 45(4) 735-744

Survey study18

Survey study

rSafety‐specific

Occupational

rxy

transformational leadership

Occupational safety

Self‐efficacy Job performance

Transformational leadership

Job performance

Kenneth Law 同济大学 2010

Meta Analysis19

Meta Analysis

R h tiResearch questionCan we combine quantitative research in the past and summarize the research findings quantitatively

Judge T Heller D Mount M (2002) Five-Factor g ( )model of personality and job satisfaction A meta-analysis Journal of Applied Psychology 87(3) 530-541

Kenneth Law 同济大学 2010

Meta analysis20

y

Big‐5 personality individualism neuroticism conscientiousness agreeableness openness to experiences

Rxy is the correlation between agreeableness and job satisfaction

Study N rxy

1 150 35 2 247 ‐07

3 386 174 124 485 278 025 76 450 29

Kenneth Law 同济大学 2010

21

Kenneth Law 同济大学 2010

Issues in survey design 22

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Issues in survey design 23

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

1 What is your research question24

1 What is your research question Is the research question testableq

改善我国旅游业未来发展的几个意见

Are the constructs well defined 「企业进取性」和企业业绩表现的关系

Do we have validated scales to measure the constructs Existing scales Western scales

Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础

Kenneth Law 同济大学 2010

What is the research question25

qIs EQ really useful in predicting work outcomes

Would a firm using a strategic human resource management approach be more competitive

What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC

Can supervisors distinguish task performance from contextual performance in the PRC

What are the antecedents of contextual performance

Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC

Kenneth Law 同济大学 2010

生涯管理要素圖26

生涯情況 信息 計劃 資源目標

探索的動機 候迭人的信息

生活的目標

1個人戰略2時間

1 解決個人問題的技能

2 控制1自我評估價值技術興趣經驗

2組織評估表現潛力分配計劃

生涯發展目標 生涯計劃執行

分配計劃

現有內部勞資市場1工作調整需要2生涯之路結構3內部提升

生涯機會信息1生涯信息系統2生涯咨詢

1未來組織經濟目標

2未來所需職工

1組織人才資源發展戰略

2重要職務分配

1工作機會2贊助人3生涯管理者

Kenneth Law 同济大学 2010

Th di ti l f LMX

27

The mediating role of LMX

OCB

TransformationalLeadership LMX-MDM

OCB

77

3280

TaskPerformance

16

Kenneth Law 同济大学 2010

Contributions28

Contributions1 Theoretical contributions

bull New variables based on theorybull New perspectivesbull New findings

h d l l b2 Methodological contributionsbull New measures (eg new scale development)

N th d ( l l h)bull New methods (eg cross‐level research)

Kenneth Law 同济大学 2010

Identifying Research Questions29

Identifying Research Questions

员 企业的交换契约

公平理论与员工公民行为

Literature

员工企业的交换契约Incremental theoretical

contribution

Research Questions

M h d l l Practitioners公司治理

Methodological rigor

59岁现象

合资企业员工本土化

Kenneth Law 同济大学 2010

What is a research question30

What is a research question

My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust

How do you like that

Expected returnknowledge sharing

knowledge sharing

Benevolence

Expected return

Trust

s a gtendency

s a gbehaviors

Kenneth Law 同济大学 2010

A conversation hellip31

A conversation hellip

Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate

ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic

ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal

Kenneth Law 同济大学 2010

A different view32

A different view

We do research because we want to know We do research because we want to know the answer hellip

Therefore before we start a study we must have a research question

We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question

Kenneth Law 同济大学 2010

A different view33

A different view

With a research question you would automatically ask for a theoryautomatically ask for a theory

B d h ld Based on your theory you would automatically develop some hypotheses

Kenneth Law 同济大学 2010

The research process34

The research process

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Trusttendency behaviors

Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g

Kenneth Law 同济大学 2010

The research process35

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

H d k th t k l d h i i t ti i

Trust

How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g

This is because helliphellip (a theory)

Kenneth Law 同济大学 2010

The research process36

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Y d th k l d h i i t ti b

Trust

You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)

Kenneth Law 同济大学 2010

The research process37

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

(1) Some people have an in‐born tendency to share with

Trust

(1) Some people have an in‐born tendency to share with others

(2) Some people would expect reciprocal sharing in the future

(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)

(4) helliphellip

Kenneth Law 同济大学 2010

(4) helliphellip

Some questions38

1 Has anyone done anything on this topic before (literature review)

2 Which one is the most important reason to explain knowledge sharing (theory)

3 Are there established management theories that can explain the phenomenon (theory)

4 Are these antecedents or causes (theory)

5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)

6 What is the most appropriate research design to answer my research questionresearch question

Kenneth Law 同济大学 2010

When to use which design39

When to use which design

1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis

Kenneth Law 同济大学 2010

Do Not start your research40

Do Not start your research helliphellip

With some hypothesized relationships

With a model

Just because no one has done that before

Only because you see a significant correlation in your datasetyour dataset

Without looking at the literature

Without a research questionq

If you do not find it interesting

If you do not believe it is important

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Examples17

Barling J LughlinC amp Kelloway EK (2002) Development and test of a model linking safety-specific transformational leadership and occupational safety Journal of Applied Psychology 87(3) 488-496

Vancouver JB Thompson CM Tischner EC amp Putka DJ (2002) Two studies examining the negative effect of self-efficacy on performance Journal of Applied Psychology y p J f App y gy87(3) 506-516

Dvir T Eden D Avoilio BJ Shamir B (2002) ImpactDvir T Eden D Avoilio BJ Shamir B (2002) Impact of transformational leadership on follower development and performance A field experiment Academy of Management Journal 45(4) 735-744

Kenneth Law 同济大学 2010

Journal 45(4) 735-744

Survey study18

Survey study

rSafety‐specific

Occupational

rxy

transformational leadership

Occupational safety

Self‐efficacy Job performance

Transformational leadership

Job performance

Kenneth Law 同济大学 2010

Meta Analysis19

Meta Analysis

R h tiResearch questionCan we combine quantitative research in the past and summarize the research findings quantitatively

Judge T Heller D Mount M (2002) Five-Factor g ( )model of personality and job satisfaction A meta-analysis Journal of Applied Psychology 87(3) 530-541

Kenneth Law 同济大学 2010

Meta analysis20

y

Big‐5 personality individualism neuroticism conscientiousness agreeableness openness to experiences

Rxy is the correlation between agreeableness and job satisfaction

Study N rxy

1 150 35 2 247 ‐07

3 386 174 124 485 278 025 76 450 29

Kenneth Law 同济大学 2010

21

Kenneth Law 同济大学 2010

Issues in survey design 22

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Issues in survey design 23

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

1 What is your research question24

1 What is your research question Is the research question testableq

改善我国旅游业未来发展的几个意见

Are the constructs well defined 「企业进取性」和企业业绩表现的关系

Do we have validated scales to measure the constructs Existing scales Western scales

Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础

Kenneth Law 同济大学 2010

What is the research question25

qIs EQ really useful in predicting work outcomes

Would a firm using a strategic human resource management approach be more competitive

What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC

Can supervisors distinguish task performance from contextual performance in the PRC

What are the antecedents of contextual performance

Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC

Kenneth Law 同济大学 2010

生涯管理要素圖26

生涯情況 信息 計劃 資源目標

探索的動機 候迭人的信息

生活的目標

1個人戰略2時間

1 解決個人問題的技能

2 控制1自我評估價值技術興趣經驗

2組織評估表現潛力分配計劃

生涯發展目標 生涯計劃執行

分配計劃

現有內部勞資市場1工作調整需要2生涯之路結構3內部提升

生涯機會信息1生涯信息系統2生涯咨詢

1未來組織經濟目標

2未來所需職工

1組織人才資源發展戰略

2重要職務分配

1工作機會2贊助人3生涯管理者

Kenneth Law 同济大学 2010

Th di ti l f LMX

27

The mediating role of LMX

OCB

TransformationalLeadership LMX-MDM

OCB

77

3280

TaskPerformance

16

Kenneth Law 同济大学 2010

Contributions28

Contributions1 Theoretical contributions

bull New variables based on theorybull New perspectivesbull New findings

h d l l b2 Methodological contributionsbull New measures (eg new scale development)

N th d ( l l h)bull New methods (eg cross‐level research)

Kenneth Law 同济大学 2010

Identifying Research Questions29

Identifying Research Questions

员 企业的交换契约

公平理论与员工公民行为

Literature

员工企业的交换契约Incremental theoretical

contribution

Research Questions

M h d l l Practitioners公司治理

Methodological rigor

59岁现象

合资企业员工本土化

Kenneth Law 同济大学 2010

What is a research question30

What is a research question

My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust

How do you like that

Expected returnknowledge sharing

knowledge sharing

Benevolence

Expected return

Trust

s a gtendency

s a gbehaviors

Kenneth Law 同济大学 2010

A conversation hellip31

A conversation hellip

Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate

ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic

ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal

Kenneth Law 同济大学 2010

A different view32

A different view

We do research because we want to know We do research because we want to know the answer hellip

Therefore before we start a study we must have a research question

We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question

Kenneth Law 同济大学 2010

A different view33

A different view

With a research question you would automatically ask for a theoryautomatically ask for a theory

B d h ld Based on your theory you would automatically develop some hypotheses

Kenneth Law 同济大学 2010

The research process34

The research process

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Trusttendency behaviors

Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g

Kenneth Law 同济大学 2010

The research process35

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

H d k th t k l d h i i t ti i

Trust

How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g

This is because helliphellip (a theory)

Kenneth Law 同济大学 2010

The research process36

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Y d th k l d h i i t ti b

Trust

You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)

Kenneth Law 同济大学 2010

The research process37

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

(1) Some people have an in‐born tendency to share with

Trust

(1) Some people have an in‐born tendency to share with others

(2) Some people would expect reciprocal sharing in the future

(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)

(4) helliphellip

Kenneth Law 同济大学 2010

(4) helliphellip

Some questions38

1 Has anyone done anything on this topic before (literature review)

2 Which one is the most important reason to explain knowledge sharing (theory)

3 Are there established management theories that can explain the phenomenon (theory)

4 Are these antecedents or causes (theory)

5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)

6 What is the most appropriate research design to answer my research questionresearch question

Kenneth Law 同济大学 2010

When to use which design39

When to use which design

1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis

Kenneth Law 同济大学 2010

Do Not start your research40

Do Not start your research helliphellip

With some hypothesized relationships

With a model

Just because no one has done that before

Only because you see a significant correlation in your datasetyour dataset

Without looking at the literature

Without a research questionq

If you do not find it interesting

If you do not believe it is important

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Survey study18

Survey study

rSafety‐specific

Occupational

rxy

transformational leadership

Occupational safety

Self‐efficacy Job performance

Transformational leadership

Job performance

Kenneth Law 同济大学 2010

Meta Analysis19

Meta Analysis

R h tiResearch questionCan we combine quantitative research in the past and summarize the research findings quantitatively

Judge T Heller D Mount M (2002) Five-Factor g ( )model of personality and job satisfaction A meta-analysis Journal of Applied Psychology 87(3) 530-541

Kenneth Law 同济大学 2010

Meta analysis20

y

Big‐5 personality individualism neuroticism conscientiousness agreeableness openness to experiences

Rxy is the correlation between agreeableness and job satisfaction

Study N rxy

1 150 35 2 247 ‐07

3 386 174 124 485 278 025 76 450 29

Kenneth Law 同济大学 2010

21

Kenneth Law 同济大学 2010

Issues in survey design 22

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Issues in survey design 23

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

1 What is your research question24

1 What is your research question Is the research question testableq

改善我国旅游业未来发展的几个意见

Are the constructs well defined 「企业进取性」和企业业绩表现的关系

Do we have validated scales to measure the constructs Existing scales Western scales

Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础

Kenneth Law 同济大学 2010

What is the research question25

qIs EQ really useful in predicting work outcomes

Would a firm using a strategic human resource management approach be more competitive

What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC

Can supervisors distinguish task performance from contextual performance in the PRC

What are the antecedents of contextual performance

Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC

Kenneth Law 同济大学 2010

生涯管理要素圖26

生涯情況 信息 計劃 資源目標

探索的動機 候迭人的信息

生活的目標

1個人戰略2時間

1 解決個人問題的技能

2 控制1自我評估價值技術興趣經驗

2組織評估表現潛力分配計劃

生涯發展目標 生涯計劃執行

分配計劃

現有內部勞資市場1工作調整需要2生涯之路結構3內部提升

生涯機會信息1生涯信息系統2生涯咨詢

1未來組織經濟目標

2未來所需職工

1組織人才資源發展戰略

2重要職務分配

1工作機會2贊助人3生涯管理者

Kenneth Law 同济大学 2010

Th di ti l f LMX

27

The mediating role of LMX

OCB

TransformationalLeadership LMX-MDM

OCB

77

3280

TaskPerformance

16

Kenneth Law 同济大学 2010

Contributions28

Contributions1 Theoretical contributions

bull New variables based on theorybull New perspectivesbull New findings

h d l l b2 Methodological contributionsbull New measures (eg new scale development)

N th d ( l l h)bull New methods (eg cross‐level research)

Kenneth Law 同济大学 2010

Identifying Research Questions29

Identifying Research Questions

员 企业的交换契约

公平理论与员工公民行为

Literature

员工企业的交换契约Incremental theoretical

contribution

Research Questions

M h d l l Practitioners公司治理

Methodological rigor

59岁现象

合资企业员工本土化

Kenneth Law 同济大学 2010

What is a research question30

What is a research question

My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust

How do you like that

Expected returnknowledge sharing

knowledge sharing

Benevolence

Expected return

Trust

s a gtendency

s a gbehaviors

Kenneth Law 同济大学 2010

A conversation hellip31

A conversation hellip

Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate

ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic

ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal

Kenneth Law 同济大学 2010

A different view32

A different view

We do research because we want to know We do research because we want to know the answer hellip

Therefore before we start a study we must have a research question

We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question

Kenneth Law 同济大学 2010

A different view33

A different view

With a research question you would automatically ask for a theoryautomatically ask for a theory

B d h ld Based on your theory you would automatically develop some hypotheses

Kenneth Law 同济大学 2010

The research process34

The research process

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Trusttendency behaviors

Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g

Kenneth Law 同济大学 2010

The research process35

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

H d k th t k l d h i i t ti i

Trust

How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g

This is because helliphellip (a theory)

Kenneth Law 同济大学 2010

The research process36

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Y d th k l d h i i t ti b

Trust

You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)

Kenneth Law 同济大学 2010

The research process37

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

(1) Some people have an in‐born tendency to share with

Trust

(1) Some people have an in‐born tendency to share with others

(2) Some people would expect reciprocal sharing in the future

(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)

(4) helliphellip

Kenneth Law 同济大学 2010

(4) helliphellip

Some questions38

1 Has anyone done anything on this topic before (literature review)

2 Which one is the most important reason to explain knowledge sharing (theory)

3 Are there established management theories that can explain the phenomenon (theory)

4 Are these antecedents or causes (theory)

5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)

6 What is the most appropriate research design to answer my research questionresearch question

Kenneth Law 同济大学 2010

When to use which design39

When to use which design

1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis

Kenneth Law 同济大学 2010

Do Not start your research40

Do Not start your research helliphellip

With some hypothesized relationships

With a model

Just because no one has done that before

Only because you see a significant correlation in your datasetyour dataset

Without looking at the literature

Without a research questionq

If you do not find it interesting

If you do not believe it is important

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Meta Analysis19

Meta Analysis

R h tiResearch questionCan we combine quantitative research in the past and summarize the research findings quantitatively

Judge T Heller D Mount M (2002) Five-Factor g ( )model of personality and job satisfaction A meta-analysis Journal of Applied Psychology 87(3) 530-541

Kenneth Law 同济大学 2010

Meta analysis20

y

Big‐5 personality individualism neuroticism conscientiousness agreeableness openness to experiences

Rxy is the correlation between agreeableness and job satisfaction

Study N rxy

1 150 35 2 247 ‐07

3 386 174 124 485 278 025 76 450 29

Kenneth Law 同济大学 2010

21

Kenneth Law 同济大学 2010

Issues in survey design 22

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Issues in survey design 23

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

1 What is your research question24

1 What is your research question Is the research question testableq

改善我国旅游业未来发展的几个意见

Are the constructs well defined 「企业进取性」和企业业绩表现的关系

Do we have validated scales to measure the constructs Existing scales Western scales

Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础

Kenneth Law 同济大学 2010

What is the research question25

qIs EQ really useful in predicting work outcomes

Would a firm using a strategic human resource management approach be more competitive

What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC

Can supervisors distinguish task performance from contextual performance in the PRC

What are the antecedents of contextual performance

Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC

Kenneth Law 同济大学 2010

生涯管理要素圖26

生涯情況 信息 計劃 資源目標

探索的動機 候迭人的信息

生活的目標

1個人戰略2時間

1 解決個人問題的技能

2 控制1自我評估價值技術興趣經驗

2組織評估表現潛力分配計劃

生涯發展目標 生涯計劃執行

分配計劃

現有內部勞資市場1工作調整需要2生涯之路結構3內部提升

生涯機會信息1生涯信息系統2生涯咨詢

1未來組織經濟目標

2未來所需職工

1組織人才資源發展戰略

2重要職務分配

1工作機會2贊助人3生涯管理者

Kenneth Law 同济大学 2010

Th di ti l f LMX

27

The mediating role of LMX

OCB

TransformationalLeadership LMX-MDM

OCB

77

3280

TaskPerformance

16

Kenneth Law 同济大学 2010

Contributions28

Contributions1 Theoretical contributions

bull New variables based on theorybull New perspectivesbull New findings

h d l l b2 Methodological contributionsbull New measures (eg new scale development)

N th d ( l l h)bull New methods (eg cross‐level research)

Kenneth Law 同济大学 2010

Identifying Research Questions29

Identifying Research Questions

员 企业的交换契约

公平理论与员工公民行为

Literature

员工企业的交换契约Incremental theoretical

contribution

Research Questions

M h d l l Practitioners公司治理

Methodological rigor

59岁现象

合资企业员工本土化

Kenneth Law 同济大学 2010

What is a research question30

What is a research question

My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust

How do you like that

Expected returnknowledge sharing

knowledge sharing

Benevolence

Expected return

Trust

s a gtendency

s a gbehaviors

Kenneth Law 同济大学 2010

A conversation hellip31

A conversation hellip

Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate

ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic

ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal

Kenneth Law 同济大学 2010

A different view32

A different view

We do research because we want to know We do research because we want to know the answer hellip

Therefore before we start a study we must have a research question

We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question

Kenneth Law 同济大学 2010

A different view33

A different view

With a research question you would automatically ask for a theoryautomatically ask for a theory

B d h ld Based on your theory you would automatically develop some hypotheses

Kenneth Law 同济大学 2010

The research process34

The research process

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Trusttendency behaviors

Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g

Kenneth Law 同济大学 2010

The research process35

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

H d k th t k l d h i i t ti i

Trust

How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g

This is because helliphellip (a theory)

Kenneth Law 同济大学 2010

The research process36

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Y d th k l d h i i t ti b

Trust

You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)

Kenneth Law 同济大学 2010

The research process37

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

(1) Some people have an in‐born tendency to share with

Trust

(1) Some people have an in‐born tendency to share with others

(2) Some people would expect reciprocal sharing in the future

(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)

(4) helliphellip

Kenneth Law 同济大学 2010

(4) helliphellip

Some questions38

1 Has anyone done anything on this topic before (literature review)

2 Which one is the most important reason to explain knowledge sharing (theory)

3 Are there established management theories that can explain the phenomenon (theory)

4 Are these antecedents or causes (theory)

5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)

6 What is the most appropriate research design to answer my research questionresearch question

Kenneth Law 同济大学 2010

When to use which design39

When to use which design

1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis

Kenneth Law 同济大学 2010

Do Not start your research40

Do Not start your research helliphellip

With some hypothesized relationships

With a model

Just because no one has done that before

Only because you see a significant correlation in your datasetyour dataset

Without looking at the literature

Without a research questionq

If you do not find it interesting

If you do not believe it is important

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Meta analysis20

y

Big‐5 personality individualism neuroticism conscientiousness agreeableness openness to experiences

Rxy is the correlation between agreeableness and job satisfaction

Study N rxy

1 150 35 2 247 ‐07

3 386 174 124 485 278 025 76 450 29

Kenneth Law 同济大学 2010

21

Kenneth Law 同济大学 2010

Issues in survey design 22

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Issues in survey design 23

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

1 What is your research question24

1 What is your research question Is the research question testableq

改善我国旅游业未来发展的几个意见

Are the constructs well defined 「企业进取性」和企业业绩表现的关系

Do we have validated scales to measure the constructs Existing scales Western scales

Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础

Kenneth Law 同济大学 2010

What is the research question25

qIs EQ really useful in predicting work outcomes

Would a firm using a strategic human resource management approach be more competitive

What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC

Can supervisors distinguish task performance from contextual performance in the PRC

What are the antecedents of contextual performance

Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC

Kenneth Law 同济大学 2010

生涯管理要素圖26

生涯情況 信息 計劃 資源目標

探索的動機 候迭人的信息

生活的目標

1個人戰略2時間

1 解決個人問題的技能

2 控制1自我評估價值技術興趣經驗

2組織評估表現潛力分配計劃

生涯發展目標 生涯計劃執行

分配計劃

現有內部勞資市場1工作調整需要2生涯之路結構3內部提升

生涯機會信息1生涯信息系統2生涯咨詢

1未來組織經濟目標

2未來所需職工

1組織人才資源發展戰略

2重要職務分配

1工作機會2贊助人3生涯管理者

Kenneth Law 同济大学 2010

Th di ti l f LMX

27

The mediating role of LMX

OCB

TransformationalLeadership LMX-MDM

OCB

77

3280

TaskPerformance

16

Kenneth Law 同济大学 2010

Contributions28

Contributions1 Theoretical contributions

bull New variables based on theorybull New perspectivesbull New findings

h d l l b2 Methodological contributionsbull New measures (eg new scale development)

N th d ( l l h)bull New methods (eg cross‐level research)

Kenneth Law 同济大学 2010

Identifying Research Questions29

Identifying Research Questions

员 企业的交换契约

公平理论与员工公民行为

Literature

员工企业的交换契约Incremental theoretical

contribution

Research Questions

M h d l l Practitioners公司治理

Methodological rigor

59岁现象

合资企业员工本土化

Kenneth Law 同济大学 2010

What is a research question30

What is a research question

My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust

How do you like that

Expected returnknowledge sharing

knowledge sharing

Benevolence

Expected return

Trust

s a gtendency

s a gbehaviors

Kenneth Law 同济大学 2010

A conversation hellip31

A conversation hellip

Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate

ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic

ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal

Kenneth Law 同济大学 2010

A different view32

A different view

We do research because we want to know We do research because we want to know the answer hellip

Therefore before we start a study we must have a research question

We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question

Kenneth Law 同济大学 2010

A different view33

A different view

With a research question you would automatically ask for a theoryautomatically ask for a theory

B d h ld Based on your theory you would automatically develop some hypotheses

Kenneth Law 同济大学 2010

The research process34

The research process

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Trusttendency behaviors

Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g

Kenneth Law 同济大学 2010

The research process35

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

H d k th t k l d h i i t ti i

Trust

How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g

This is because helliphellip (a theory)

Kenneth Law 同济大学 2010

The research process36

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Y d th k l d h i i t ti b

Trust

You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)

Kenneth Law 同济大学 2010

The research process37

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

(1) Some people have an in‐born tendency to share with

Trust

(1) Some people have an in‐born tendency to share with others

(2) Some people would expect reciprocal sharing in the future

(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)

(4) helliphellip

Kenneth Law 同济大学 2010

(4) helliphellip

Some questions38

1 Has anyone done anything on this topic before (literature review)

2 Which one is the most important reason to explain knowledge sharing (theory)

3 Are there established management theories that can explain the phenomenon (theory)

4 Are these antecedents or causes (theory)

5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)

6 What is the most appropriate research design to answer my research questionresearch question

Kenneth Law 同济大学 2010

When to use which design39

When to use which design

1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis

Kenneth Law 同济大学 2010

Do Not start your research40

Do Not start your research helliphellip

With some hypothesized relationships

With a model

Just because no one has done that before

Only because you see a significant correlation in your datasetyour dataset

Without looking at the literature

Without a research questionq

If you do not find it interesting

If you do not believe it is important

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

21

Kenneth Law 同济大学 2010

Issues in survey design 22

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Issues in survey design 23

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

1 What is your research question24

1 What is your research question Is the research question testableq

改善我国旅游业未来发展的几个意见

Are the constructs well defined 「企业进取性」和企业业绩表现的关系

Do we have validated scales to measure the constructs Existing scales Western scales

Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础

Kenneth Law 同济大学 2010

What is the research question25

qIs EQ really useful in predicting work outcomes

Would a firm using a strategic human resource management approach be more competitive

What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC

Can supervisors distinguish task performance from contextual performance in the PRC

What are the antecedents of contextual performance

Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC

Kenneth Law 同济大学 2010

生涯管理要素圖26

生涯情況 信息 計劃 資源目標

探索的動機 候迭人的信息

生活的目標

1個人戰略2時間

1 解決個人問題的技能

2 控制1自我評估價值技術興趣經驗

2組織評估表現潛力分配計劃

生涯發展目標 生涯計劃執行

分配計劃

現有內部勞資市場1工作調整需要2生涯之路結構3內部提升

生涯機會信息1生涯信息系統2生涯咨詢

1未來組織經濟目標

2未來所需職工

1組織人才資源發展戰略

2重要職務分配

1工作機會2贊助人3生涯管理者

Kenneth Law 同济大学 2010

Th di ti l f LMX

27

The mediating role of LMX

OCB

TransformationalLeadership LMX-MDM

OCB

77

3280

TaskPerformance

16

Kenneth Law 同济大学 2010

Contributions28

Contributions1 Theoretical contributions

bull New variables based on theorybull New perspectivesbull New findings

h d l l b2 Methodological contributionsbull New measures (eg new scale development)

N th d ( l l h)bull New methods (eg cross‐level research)

Kenneth Law 同济大学 2010

Identifying Research Questions29

Identifying Research Questions

员 企业的交换契约

公平理论与员工公民行为

Literature

员工企业的交换契约Incremental theoretical

contribution

Research Questions

M h d l l Practitioners公司治理

Methodological rigor

59岁现象

合资企业员工本土化

Kenneth Law 同济大学 2010

What is a research question30

What is a research question

My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust

How do you like that

Expected returnknowledge sharing

knowledge sharing

Benevolence

Expected return

Trust

s a gtendency

s a gbehaviors

Kenneth Law 同济大学 2010

A conversation hellip31

A conversation hellip

Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate

ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic

ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal

Kenneth Law 同济大学 2010

A different view32

A different view

We do research because we want to know We do research because we want to know the answer hellip

Therefore before we start a study we must have a research question

We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question

Kenneth Law 同济大学 2010

A different view33

A different view

With a research question you would automatically ask for a theoryautomatically ask for a theory

B d h ld Based on your theory you would automatically develop some hypotheses

Kenneth Law 同济大学 2010

The research process34

The research process

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Trusttendency behaviors

Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g

Kenneth Law 同济大学 2010

The research process35

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

H d k th t k l d h i i t ti i

Trust

How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g

This is because helliphellip (a theory)

Kenneth Law 同济大学 2010

The research process36

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Y d th k l d h i i t ti b

Trust

You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)

Kenneth Law 同济大学 2010

The research process37

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

(1) Some people have an in‐born tendency to share with

Trust

(1) Some people have an in‐born tendency to share with others

(2) Some people would expect reciprocal sharing in the future

(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)

(4) helliphellip

Kenneth Law 同济大学 2010

(4) helliphellip

Some questions38

1 Has anyone done anything on this topic before (literature review)

2 Which one is the most important reason to explain knowledge sharing (theory)

3 Are there established management theories that can explain the phenomenon (theory)

4 Are these antecedents or causes (theory)

5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)

6 What is the most appropriate research design to answer my research questionresearch question

Kenneth Law 同济大学 2010

When to use which design39

When to use which design

1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis

Kenneth Law 同济大学 2010

Do Not start your research40

Do Not start your research helliphellip

With some hypothesized relationships

With a model

Just because no one has done that before

Only because you see a significant correlation in your datasetyour dataset

Without looking at the literature

Without a research questionq

If you do not find it interesting

If you do not believe it is important

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Issues in survey design 22

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Issues in survey design 23

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

1 What is your research question24

1 What is your research question Is the research question testableq

改善我国旅游业未来发展的几个意见

Are the constructs well defined 「企业进取性」和企业业绩表现的关系

Do we have validated scales to measure the constructs Existing scales Western scales

Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础

Kenneth Law 同济大学 2010

What is the research question25

qIs EQ really useful in predicting work outcomes

Would a firm using a strategic human resource management approach be more competitive

What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC

Can supervisors distinguish task performance from contextual performance in the PRC

What are the antecedents of contextual performance

Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC

Kenneth Law 同济大学 2010

生涯管理要素圖26

生涯情況 信息 計劃 資源目標

探索的動機 候迭人的信息

生活的目標

1個人戰略2時間

1 解決個人問題的技能

2 控制1自我評估價值技術興趣經驗

2組織評估表現潛力分配計劃

生涯發展目標 生涯計劃執行

分配計劃

現有內部勞資市場1工作調整需要2生涯之路結構3內部提升

生涯機會信息1生涯信息系統2生涯咨詢

1未來組織經濟目標

2未來所需職工

1組織人才資源發展戰略

2重要職務分配

1工作機會2贊助人3生涯管理者

Kenneth Law 同济大学 2010

Th di ti l f LMX

27

The mediating role of LMX

OCB

TransformationalLeadership LMX-MDM

OCB

77

3280

TaskPerformance

16

Kenneth Law 同济大学 2010

Contributions28

Contributions1 Theoretical contributions

bull New variables based on theorybull New perspectivesbull New findings

h d l l b2 Methodological contributionsbull New measures (eg new scale development)

N th d ( l l h)bull New methods (eg cross‐level research)

Kenneth Law 同济大学 2010

Identifying Research Questions29

Identifying Research Questions

员 企业的交换契约

公平理论与员工公民行为

Literature

员工企业的交换契约Incremental theoretical

contribution

Research Questions

M h d l l Practitioners公司治理

Methodological rigor

59岁现象

合资企业员工本土化

Kenneth Law 同济大学 2010

What is a research question30

What is a research question

My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust

How do you like that

Expected returnknowledge sharing

knowledge sharing

Benevolence

Expected return

Trust

s a gtendency

s a gbehaviors

Kenneth Law 同济大学 2010

A conversation hellip31

A conversation hellip

Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate

ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic

ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal

Kenneth Law 同济大学 2010

A different view32

A different view

We do research because we want to know We do research because we want to know the answer hellip

Therefore before we start a study we must have a research question

We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question

Kenneth Law 同济大学 2010

A different view33

A different view

With a research question you would automatically ask for a theoryautomatically ask for a theory

B d h ld Based on your theory you would automatically develop some hypotheses

Kenneth Law 同济大学 2010

The research process34

The research process

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Trusttendency behaviors

Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g

Kenneth Law 同济大学 2010

The research process35

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

H d k th t k l d h i i t ti i

Trust

How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g

This is because helliphellip (a theory)

Kenneth Law 同济大学 2010

The research process36

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Y d th k l d h i i t ti b

Trust

You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)

Kenneth Law 同济大学 2010

The research process37

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

(1) Some people have an in‐born tendency to share with

Trust

(1) Some people have an in‐born tendency to share with others

(2) Some people would expect reciprocal sharing in the future

(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)

(4) helliphellip

Kenneth Law 同济大学 2010

(4) helliphellip

Some questions38

1 Has anyone done anything on this topic before (literature review)

2 Which one is the most important reason to explain knowledge sharing (theory)

3 Are there established management theories that can explain the phenomenon (theory)

4 Are these antecedents or causes (theory)

5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)

6 What is the most appropriate research design to answer my research questionresearch question

Kenneth Law 同济大学 2010

When to use which design39

When to use which design

1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis

Kenneth Law 同济大学 2010

Do Not start your research40

Do Not start your research helliphellip

With some hypothesized relationships

With a model

Just because no one has done that before

Only because you see a significant correlation in your datasetyour dataset

Without looking at the literature

Without a research questionq

If you do not find it interesting

If you do not believe it is important

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Issues in survey design 23

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

1 What is your research question24

1 What is your research question Is the research question testableq

改善我国旅游业未来发展的几个意见

Are the constructs well defined 「企业进取性」和企业业绩表现的关系

Do we have validated scales to measure the constructs Existing scales Western scales

Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础

Kenneth Law 同济大学 2010

What is the research question25

qIs EQ really useful in predicting work outcomes

Would a firm using a strategic human resource management approach be more competitive

What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC

Can supervisors distinguish task performance from contextual performance in the PRC

What are the antecedents of contextual performance

Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC

Kenneth Law 同济大学 2010

生涯管理要素圖26

生涯情況 信息 計劃 資源目標

探索的動機 候迭人的信息

生活的目標

1個人戰略2時間

1 解決個人問題的技能

2 控制1自我評估價值技術興趣經驗

2組織評估表現潛力分配計劃

生涯發展目標 生涯計劃執行

分配計劃

現有內部勞資市場1工作調整需要2生涯之路結構3內部提升

生涯機會信息1生涯信息系統2生涯咨詢

1未來組織經濟目標

2未來所需職工

1組織人才資源發展戰略

2重要職務分配

1工作機會2贊助人3生涯管理者

Kenneth Law 同济大学 2010

Th di ti l f LMX

27

The mediating role of LMX

OCB

TransformationalLeadership LMX-MDM

OCB

77

3280

TaskPerformance

16

Kenneth Law 同济大学 2010

Contributions28

Contributions1 Theoretical contributions

bull New variables based on theorybull New perspectivesbull New findings

h d l l b2 Methodological contributionsbull New measures (eg new scale development)

N th d ( l l h)bull New methods (eg cross‐level research)

Kenneth Law 同济大学 2010

Identifying Research Questions29

Identifying Research Questions

员 企业的交换契约

公平理论与员工公民行为

Literature

员工企业的交换契约Incremental theoretical

contribution

Research Questions

M h d l l Practitioners公司治理

Methodological rigor

59岁现象

合资企业员工本土化

Kenneth Law 同济大学 2010

What is a research question30

What is a research question

My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust

How do you like that

Expected returnknowledge sharing

knowledge sharing

Benevolence

Expected return

Trust

s a gtendency

s a gbehaviors

Kenneth Law 同济大学 2010

A conversation hellip31

A conversation hellip

Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate

ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic

ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal

Kenneth Law 同济大学 2010

A different view32

A different view

We do research because we want to know We do research because we want to know the answer hellip

Therefore before we start a study we must have a research question

We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question

Kenneth Law 同济大学 2010

A different view33

A different view

With a research question you would automatically ask for a theoryautomatically ask for a theory

B d h ld Based on your theory you would automatically develop some hypotheses

Kenneth Law 同济大学 2010

The research process34

The research process

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Trusttendency behaviors

Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g

Kenneth Law 同济大学 2010

The research process35

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

H d k th t k l d h i i t ti i

Trust

How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g

This is because helliphellip (a theory)

Kenneth Law 同济大学 2010

The research process36

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Y d th k l d h i i t ti b

Trust

You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)

Kenneth Law 同济大学 2010

The research process37

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

(1) Some people have an in‐born tendency to share with

Trust

(1) Some people have an in‐born tendency to share with others

(2) Some people would expect reciprocal sharing in the future

(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)

(4) helliphellip

Kenneth Law 同济大学 2010

(4) helliphellip

Some questions38

1 Has anyone done anything on this topic before (literature review)

2 Which one is the most important reason to explain knowledge sharing (theory)

3 Are there established management theories that can explain the phenomenon (theory)

4 Are these antecedents or causes (theory)

5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)

6 What is the most appropriate research design to answer my research questionresearch question

Kenneth Law 同济大学 2010

When to use which design39

When to use which design

1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis

Kenneth Law 同济大学 2010

Do Not start your research40

Do Not start your research helliphellip

With some hypothesized relationships

With a model

Just because no one has done that before

Only because you see a significant correlation in your datasetyour dataset

Without looking at the literature

Without a research questionq

If you do not find it interesting

If you do not believe it is important

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

1 What is your research question24

1 What is your research question Is the research question testableq

改善我国旅游业未来发展的几个意见

Are the constructs well defined 「企业进取性」和企业业绩表现的关系

Do we have validated scales to measure the constructs Existing scales Western scales

Are the relationships well justified Are the relationships well justified 你有没有现存的理论基础

Kenneth Law 同济大学 2010

What is the research question25

qIs EQ really useful in predicting work outcomes

Would a firm using a strategic human resource management approach be more competitive

What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC

Can supervisors distinguish task performance from contextual performance in the PRC

What are the antecedents of contextual performance

Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC

Kenneth Law 同济大学 2010

生涯管理要素圖26

生涯情況 信息 計劃 資源目標

探索的動機 候迭人的信息

生活的目標

1個人戰略2時間

1 解決個人問題的技能

2 控制1自我評估價值技術興趣經驗

2組織評估表現潛力分配計劃

生涯發展目標 生涯計劃執行

分配計劃

現有內部勞資市場1工作調整需要2生涯之路結構3內部提升

生涯機會信息1生涯信息系統2生涯咨詢

1未來組織經濟目標

2未來所需職工

1組織人才資源發展戰略

2重要職務分配

1工作機會2贊助人3生涯管理者

Kenneth Law 同济大学 2010

Th di ti l f LMX

27

The mediating role of LMX

OCB

TransformationalLeadership LMX-MDM

OCB

77

3280

TaskPerformance

16

Kenneth Law 同济大学 2010

Contributions28

Contributions1 Theoretical contributions

bull New variables based on theorybull New perspectivesbull New findings

h d l l b2 Methodological contributionsbull New measures (eg new scale development)

N th d ( l l h)bull New methods (eg cross‐level research)

Kenneth Law 同济大学 2010

Identifying Research Questions29

Identifying Research Questions

员 企业的交换契约

公平理论与员工公民行为

Literature

员工企业的交换契约Incremental theoretical

contribution

Research Questions

M h d l l Practitioners公司治理

Methodological rigor

59岁现象

合资企业员工本土化

Kenneth Law 同济大学 2010

What is a research question30

What is a research question

My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust

How do you like that

Expected returnknowledge sharing

knowledge sharing

Benevolence

Expected return

Trust

s a gtendency

s a gbehaviors

Kenneth Law 同济大学 2010

A conversation hellip31

A conversation hellip

Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate

ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic

ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal

Kenneth Law 同济大学 2010

A different view32

A different view

We do research because we want to know We do research because we want to know the answer hellip

Therefore before we start a study we must have a research question

We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question

Kenneth Law 同济大学 2010

A different view33

A different view

With a research question you would automatically ask for a theoryautomatically ask for a theory

B d h ld Based on your theory you would automatically develop some hypotheses

Kenneth Law 同济大学 2010

The research process34

The research process

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Trusttendency behaviors

Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g

Kenneth Law 同济大学 2010

The research process35

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

H d k th t k l d h i i t ti i

Trust

How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g

This is because helliphellip (a theory)

Kenneth Law 同济大学 2010

The research process36

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Y d th k l d h i i t ti b

Trust

You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)

Kenneth Law 同济大学 2010

The research process37

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

(1) Some people have an in‐born tendency to share with

Trust

(1) Some people have an in‐born tendency to share with others

(2) Some people would expect reciprocal sharing in the future

(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)

(4) helliphellip

Kenneth Law 同济大学 2010

(4) helliphellip

Some questions38

1 Has anyone done anything on this topic before (literature review)

2 Which one is the most important reason to explain knowledge sharing (theory)

3 Are there established management theories that can explain the phenomenon (theory)

4 Are these antecedents or causes (theory)

5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)

6 What is the most appropriate research design to answer my research questionresearch question

Kenneth Law 同济大学 2010

When to use which design39

When to use which design

1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis

Kenneth Law 同济大学 2010

Do Not start your research40

Do Not start your research helliphellip

With some hypothesized relationships

With a model

Just because no one has done that before

Only because you see a significant correlation in your datasetyour dataset

Without looking at the literature

Without a research questionq

If you do not find it interesting

If you do not believe it is important

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

What is the research question25

qIs EQ really useful in predicting work outcomes

Would a firm using a strategic human resource management approach be more competitive

What are the factors affecting localization of expatriate position inWhat are the factors affecting localization of expatriate position in the PRC

Can supervisors distinguish task performance from contextual performance in the PRC

What are the antecedents of contextual performance

Who would work in JVs in China and what are the factors leading to career success in foreign firms in the PRC

Kenneth Law 同济大学 2010

生涯管理要素圖26

生涯情況 信息 計劃 資源目標

探索的動機 候迭人的信息

生活的目標

1個人戰略2時間

1 解決個人問題的技能

2 控制1自我評估價值技術興趣經驗

2組織評估表現潛力分配計劃

生涯發展目標 生涯計劃執行

分配計劃

現有內部勞資市場1工作調整需要2生涯之路結構3內部提升

生涯機會信息1生涯信息系統2生涯咨詢

1未來組織經濟目標

2未來所需職工

1組織人才資源發展戰略

2重要職務分配

1工作機會2贊助人3生涯管理者

Kenneth Law 同济大学 2010

Th di ti l f LMX

27

The mediating role of LMX

OCB

TransformationalLeadership LMX-MDM

OCB

77

3280

TaskPerformance

16

Kenneth Law 同济大学 2010

Contributions28

Contributions1 Theoretical contributions

bull New variables based on theorybull New perspectivesbull New findings

h d l l b2 Methodological contributionsbull New measures (eg new scale development)

N th d ( l l h)bull New methods (eg cross‐level research)

Kenneth Law 同济大学 2010

Identifying Research Questions29

Identifying Research Questions

员 企业的交换契约

公平理论与员工公民行为

Literature

员工企业的交换契约Incremental theoretical

contribution

Research Questions

M h d l l Practitioners公司治理

Methodological rigor

59岁现象

合资企业员工本土化

Kenneth Law 同济大学 2010

What is a research question30

What is a research question

My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust

How do you like that

Expected returnknowledge sharing

knowledge sharing

Benevolence

Expected return

Trust

s a gtendency

s a gbehaviors

Kenneth Law 同济大学 2010

A conversation hellip31

A conversation hellip

Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate

ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic

ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal

Kenneth Law 同济大学 2010

A different view32

A different view

We do research because we want to know We do research because we want to know the answer hellip

Therefore before we start a study we must have a research question

We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question

Kenneth Law 同济大学 2010

A different view33

A different view

With a research question you would automatically ask for a theoryautomatically ask for a theory

B d h ld Based on your theory you would automatically develop some hypotheses

Kenneth Law 同济大学 2010

The research process34

The research process

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Trusttendency behaviors

Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g

Kenneth Law 同济大学 2010

The research process35

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

H d k th t k l d h i i t ti i

Trust

How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g

This is because helliphellip (a theory)

Kenneth Law 同济大学 2010

The research process36

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Y d th k l d h i i t ti b

Trust

You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)

Kenneth Law 同济大学 2010

The research process37

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

(1) Some people have an in‐born tendency to share with

Trust

(1) Some people have an in‐born tendency to share with others

(2) Some people would expect reciprocal sharing in the future

(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)

(4) helliphellip

Kenneth Law 同济大学 2010

(4) helliphellip

Some questions38

1 Has anyone done anything on this topic before (literature review)

2 Which one is the most important reason to explain knowledge sharing (theory)

3 Are there established management theories that can explain the phenomenon (theory)

4 Are these antecedents or causes (theory)

5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)

6 What is the most appropriate research design to answer my research questionresearch question

Kenneth Law 同济大学 2010

When to use which design39

When to use which design

1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis

Kenneth Law 同济大学 2010

Do Not start your research40

Do Not start your research helliphellip

With some hypothesized relationships

With a model

Just because no one has done that before

Only because you see a significant correlation in your datasetyour dataset

Without looking at the literature

Without a research questionq

If you do not find it interesting

If you do not believe it is important

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

生涯管理要素圖26

生涯情況 信息 計劃 資源目標

探索的動機 候迭人的信息

生活的目標

1個人戰略2時間

1 解決個人問題的技能

2 控制1自我評估價值技術興趣經驗

2組織評估表現潛力分配計劃

生涯發展目標 生涯計劃執行

分配計劃

現有內部勞資市場1工作調整需要2生涯之路結構3內部提升

生涯機會信息1生涯信息系統2生涯咨詢

1未來組織經濟目標

2未來所需職工

1組織人才資源發展戰略

2重要職務分配

1工作機會2贊助人3生涯管理者

Kenneth Law 同济大学 2010

Th di ti l f LMX

27

The mediating role of LMX

OCB

TransformationalLeadership LMX-MDM

OCB

77

3280

TaskPerformance

16

Kenneth Law 同济大学 2010

Contributions28

Contributions1 Theoretical contributions

bull New variables based on theorybull New perspectivesbull New findings

h d l l b2 Methodological contributionsbull New measures (eg new scale development)

N th d ( l l h)bull New methods (eg cross‐level research)

Kenneth Law 同济大学 2010

Identifying Research Questions29

Identifying Research Questions

员 企业的交换契约

公平理论与员工公民行为

Literature

员工企业的交换契约Incremental theoretical

contribution

Research Questions

M h d l l Practitioners公司治理

Methodological rigor

59岁现象

合资企业员工本土化

Kenneth Law 同济大学 2010

What is a research question30

What is a research question

My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust

How do you like that

Expected returnknowledge sharing

knowledge sharing

Benevolence

Expected return

Trust

s a gtendency

s a gbehaviors

Kenneth Law 同济大学 2010

A conversation hellip31

A conversation hellip

Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate

ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic

ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal

Kenneth Law 同济大学 2010

A different view32

A different view

We do research because we want to know We do research because we want to know the answer hellip

Therefore before we start a study we must have a research question

We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question

Kenneth Law 同济大学 2010

A different view33

A different view

With a research question you would automatically ask for a theoryautomatically ask for a theory

B d h ld Based on your theory you would automatically develop some hypotheses

Kenneth Law 同济大学 2010

The research process34

The research process

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Trusttendency behaviors

Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g

Kenneth Law 同济大学 2010

The research process35

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

H d k th t k l d h i i t ti i

Trust

How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g

This is because helliphellip (a theory)

Kenneth Law 同济大学 2010

The research process36

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Y d th k l d h i i t ti b

Trust

You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)

Kenneth Law 同济大学 2010

The research process37

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

(1) Some people have an in‐born tendency to share with

Trust

(1) Some people have an in‐born tendency to share with others

(2) Some people would expect reciprocal sharing in the future

(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)

(4) helliphellip

Kenneth Law 同济大学 2010

(4) helliphellip

Some questions38

1 Has anyone done anything on this topic before (literature review)

2 Which one is the most important reason to explain knowledge sharing (theory)

3 Are there established management theories that can explain the phenomenon (theory)

4 Are these antecedents or causes (theory)

5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)

6 What is the most appropriate research design to answer my research questionresearch question

Kenneth Law 同济大学 2010

When to use which design39

When to use which design

1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis

Kenneth Law 同济大学 2010

Do Not start your research40

Do Not start your research helliphellip

With some hypothesized relationships

With a model

Just because no one has done that before

Only because you see a significant correlation in your datasetyour dataset

Without looking at the literature

Without a research questionq

If you do not find it interesting

If you do not believe it is important

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Th di ti l f LMX

27

The mediating role of LMX

OCB

TransformationalLeadership LMX-MDM

OCB

77

3280

TaskPerformance

16

Kenneth Law 同济大学 2010

Contributions28

Contributions1 Theoretical contributions

bull New variables based on theorybull New perspectivesbull New findings

h d l l b2 Methodological contributionsbull New measures (eg new scale development)

N th d ( l l h)bull New methods (eg cross‐level research)

Kenneth Law 同济大学 2010

Identifying Research Questions29

Identifying Research Questions

员 企业的交换契约

公平理论与员工公民行为

Literature

员工企业的交换契约Incremental theoretical

contribution

Research Questions

M h d l l Practitioners公司治理

Methodological rigor

59岁现象

合资企业员工本土化

Kenneth Law 同济大学 2010

What is a research question30

What is a research question

My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust

How do you like that

Expected returnknowledge sharing

knowledge sharing

Benevolence

Expected return

Trust

s a gtendency

s a gbehaviors

Kenneth Law 同济大学 2010

A conversation hellip31

A conversation hellip

Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate

ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic

ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal

Kenneth Law 同济大学 2010

A different view32

A different view

We do research because we want to know We do research because we want to know the answer hellip

Therefore before we start a study we must have a research question

We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question

Kenneth Law 同济大学 2010

A different view33

A different view

With a research question you would automatically ask for a theoryautomatically ask for a theory

B d h ld Based on your theory you would automatically develop some hypotheses

Kenneth Law 同济大学 2010

The research process34

The research process

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Trusttendency behaviors

Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g

Kenneth Law 同济大学 2010

The research process35

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

H d k th t k l d h i i t ti i

Trust

How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g

This is because helliphellip (a theory)

Kenneth Law 同济大学 2010

The research process36

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Y d th k l d h i i t ti b

Trust

You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)

Kenneth Law 同济大学 2010

The research process37

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

(1) Some people have an in‐born tendency to share with

Trust

(1) Some people have an in‐born tendency to share with others

(2) Some people would expect reciprocal sharing in the future

(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)

(4) helliphellip

Kenneth Law 同济大学 2010

(4) helliphellip

Some questions38

1 Has anyone done anything on this topic before (literature review)

2 Which one is the most important reason to explain knowledge sharing (theory)

3 Are there established management theories that can explain the phenomenon (theory)

4 Are these antecedents or causes (theory)

5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)

6 What is the most appropriate research design to answer my research questionresearch question

Kenneth Law 同济大学 2010

When to use which design39

When to use which design

1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis

Kenneth Law 同济大学 2010

Do Not start your research40

Do Not start your research helliphellip

With some hypothesized relationships

With a model

Just because no one has done that before

Only because you see a significant correlation in your datasetyour dataset

Without looking at the literature

Without a research questionq

If you do not find it interesting

If you do not believe it is important

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Contributions28

Contributions1 Theoretical contributions

bull New variables based on theorybull New perspectivesbull New findings

h d l l b2 Methodological contributionsbull New measures (eg new scale development)

N th d ( l l h)bull New methods (eg cross‐level research)

Kenneth Law 同济大学 2010

Identifying Research Questions29

Identifying Research Questions

员 企业的交换契约

公平理论与员工公民行为

Literature

员工企业的交换契约Incremental theoretical

contribution

Research Questions

M h d l l Practitioners公司治理

Methodological rigor

59岁现象

合资企业员工本土化

Kenneth Law 同济大学 2010

What is a research question30

What is a research question

My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust

How do you like that

Expected returnknowledge sharing

knowledge sharing

Benevolence

Expected return

Trust

s a gtendency

s a gbehaviors

Kenneth Law 同济大学 2010

A conversation hellip31

A conversation hellip

Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate

ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic

ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal

Kenneth Law 同济大学 2010

A different view32

A different view

We do research because we want to know We do research because we want to know the answer hellip

Therefore before we start a study we must have a research question

We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question

Kenneth Law 同济大学 2010

A different view33

A different view

With a research question you would automatically ask for a theoryautomatically ask for a theory

B d h ld Based on your theory you would automatically develop some hypotheses

Kenneth Law 同济大学 2010

The research process34

The research process

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Trusttendency behaviors

Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g

Kenneth Law 同济大学 2010

The research process35

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

H d k th t k l d h i i t ti i

Trust

How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g

This is because helliphellip (a theory)

Kenneth Law 同济大学 2010

The research process36

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Y d th k l d h i i t ti b

Trust

You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)

Kenneth Law 同济大学 2010

The research process37

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

(1) Some people have an in‐born tendency to share with

Trust

(1) Some people have an in‐born tendency to share with others

(2) Some people would expect reciprocal sharing in the future

(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)

(4) helliphellip

Kenneth Law 同济大学 2010

(4) helliphellip

Some questions38

1 Has anyone done anything on this topic before (literature review)

2 Which one is the most important reason to explain knowledge sharing (theory)

3 Are there established management theories that can explain the phenomenon (theory)

4 Are these antecedents or causes (theory)

5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)

6 What is the most appropriate research design to answer my research questionresearch question

Kenneth Law 同济大学 2010

When to use which design39

When to use which design

1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis

Kenneth Law 同济大学 2010

Do Not start your research40

Do Not start your research helliphellip

With some hypothesized relationships

With a model

Just because no one has done that before

Only because you see a significant correlation in your datasetyour dataset

Without looking at the literature

Without a research questionq

If you do not find it interesting

If you do not believe it is important

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Identifying Research Questions29

Identifying Research Questions

员 企业的交换契约

公平理论与员工公民行为

Literature

员工企业的交换契约Incremental theoretical

contribution

Research Questions

M h d l l Practitioners公司治理

Methodological rigor

59岁现象

合资企业员工本土化

Kenneth Law 同济大学 2010

What is a research question30

What is a research question

My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust

How do you like that

Expected returnknowledge sharing

knowledge sharing

Benevolence

Expected return

Trust

s a gtendency

s a gbehaviors

Kenneth Law 同济大学 2010

A conversation hellip31

A conversation hellip

Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate

ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic

ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal

Kenneth Law 同济大学 2010

A different view32

A different view

We do research because we want to know We do research because we want to know the answer hellip

Therefore before we start a study we must have a research question

We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question

Kenneth Law 同济大学 2010

A different view33

A different view

With a research question you would automatically ask for a theoryautomatically ask for a theory

B d h ld Based on your theory you would automatically develop some hypotheses

Kenneth Law 同济大学 2010

The research process34

The research process

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Trusttendency behaviors

Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g

Kenneth Law 同济大学 2010

The research process35

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

H d k th t k l d h i i t ti i

Trust

How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g

This is because helliphellip (a theory)

Kenneth Law 同济大学 2010

The research process36

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Y d th k l d h i i t ti b

Trust

You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)

Kenneth Law 同济大学 2010

The research process37

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

(1) Some people have an in‐born tendency to share with

Trust

(1) Some people have an in‐born tendency to share with others

(2) Some people would expect reciprocal sharing in the future

(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)

(4) helliphellip

Kenneth Law 同济大学 2010

(4) helliphellip

Some questions38

1 Has anyone done anything on this topic before (literature review)

2 Which one is the most important reason to explain knowledge sharing (theory)

3 Are there established management theories that can explain the phenomenon (theory)

4 Are these antecedents or causes (theory)

5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)

6 What is the most appropriate research design to answer my research questionresearch question

Kenneth Law 同济大学 2010

When to use which design39

When to use which design

1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis

Kenneth Law 同济大学 2010

Do Not start your research40

Do Not start your research helliphellip

With some hypothesized relationships

With a model

Just because no one has done that before

Only because you see a significant correlation in your datasetyour dataset

Without looking at the literature

Without a research questionq

If you do not find it interesting

If you do not believe it is important

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

What is a research question30

What is a research question

My model has two variables They are labeled as ldquoknowledge My model has two variables They are labeled as knowledge sharing tendencyrdquo and ldquoknowledge sharing behaviorsrdquo There are three independent variables benevolence expected return and trust

How do you like that

Expected returnknowledge sharing

knowledge sharing

Benevolence

Expected return

Trust

s a gtendency

s a gbehaviors

Kenneth Law 同济大学 2010

A conversation hellip31

A conversation hellip

Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate

ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic

ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal

Kenneth Law 同济大学 2010

A different view32

A different view

We do research because we want to know We do research because we want to know the answer hellip

Therefore before we start a study we must have a research question

We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question

Kenneth Law 同济大学 2010

A different view33

A different view

With a research question you would automatically ask for a theoryautomatically ask for a theory

B d h ld Based on your theory you would automatically develop some hypotheses

Kenneth Law 同济大学 2010

The research process34

The research process

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Trusttendency behaviors

Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g

Kenneth Law 同济大学 2010

The research process35

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

H d k th t k l d h i i t ti i

Trust

How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g

This is because helliphellip (a theory)

Kenneth Law 同济大学 2010

The research process36

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Y d th k l d h i i t ti b

Trust

You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)

Kenneth Law 同济大学 2010

The research process37

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

(1) Some people have an in‐born tendency to share with

Trust

(1) Some people have an in‐born tendency to share with others

(2) Some people would expect reciprocal sharing in the future

(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)

(4) helliphellip

Kenneth Law 同济大学 2010

(4) helliphellip

Some questions38

1 Has anyone done anything on this topic before (literature review)

2 Which one is the most important reason to explain knowledge sharing (theory)

3 Are there established management theories that can explain the phenomenon (theory)

4 Are these antecedents or causes (theory)

5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)

6 What is the most appropriate research design to answer my research questionresearch question

Kenneth Law 同济大学 2010

When to use which design39

When to use which design

1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis

Kenneth Law 同济大学 2010

Do Not start your research40

Do Not start your research helliphellip

With some hypothesized relationships

With a model

Just because no one has done that before

Only because you see a significant correlation in your datasetyour dataset

Without looking at the literature

Without a research questionq

If you do not find it interesting

If you do not believe it is important

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

A conversation hellip31

A conversation hellip

Professor Why would you do this piece of researchProfessor Why would you do this piece of researchStudentBecause I have to publish a paper before I graduate

ProfessorThen why choose knowledge sharingSt d tI j t l d b t thi t iStudentI just learned about this topic

ProfessorWhat results do you expectProfessor What results do you expectStudentpublish a paper in a respectable international journal

Kenneth Law 同济大学 2010

A different view32

A different view

We do research because we want to know We do research because we want to know the answer hellip

Therefore before we start a study we must have a research question

We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question

Kenneth Law 同济大学 2010

A different view33

A different view

With a research question you would automatically ask for a theoryautomatically ask for a theory

B d h ld Based on your theory you would automatically develop some hypotheses

Kenneth Law 同济大学 2010

The research process34

The research process

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Trusttendency behaviors

Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g

Kenneth Law 同济大学 2010

The research process35

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

H d k th t k l d h i i t ti i

Trust

How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g

This is because helliphellip (a theory)

Kenneth Law 同济大学 2010

The research process36

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Y d th k l d h i i t ti b

Trust

You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)

Kenneth Law 同济大学 2010

The research process37

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

(1) Some people have an in‐born tendency to share with

Trust

(1) Some people have an in‐born tendency to share with others

(2) Some people would expect reciprocal sharing in the future

(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)

(4) helliphellip

Kenneth Law 同济大学 2010

(4) helliphellip

Some questions38

1 Has anyone done anything on this topic before (literature review)

2 Which one is the most important reason to explain knowledge sharing (theory)

3 Are there established management theories that can explain the phenomenon (theory)

4 Are these antecedents or causes (theory)

5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)

6 What is the most appropriate research design to answer my research questionresearch question

Kenneth Law 同济大学 2010

When to use which design39

When to use which design

1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis

Kenneth Law 同济大学 2010

Do Not start your research40

Do Not start your research helliphellip

With some hypothesized relationships

With a model

Just because no one has done that before

Only because you see a significant correlation in your datasetyour dataset

Without looking at the literature

Without a research questionq

If you do not find it interesting

If you do not believe it is important

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

A different view32

A different view

We do research because we want to know We do research because we want to know the answer hellip

Therefore before we start a study we must have a research question

We do not start with a topic an area or a phenomenon We start the research phenomenon We start the research process with a research question

Kenneth Law 同济大学 2010

A different view33

A different view

With a research question you would automatically ask for a theoryautomatically ask for a theory

B d h ld Based on your theory you would automatically develop some hypotheses

Kenneth Law 同济大学 2010

The research process34

The research process

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Trusttendency behaviors

Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g

Kenneth Law 同济大学 2010

The research process35

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

H d k th t k l d h i i t ti i

Trust

How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g

This is because helliphellip (a theory)

Kenneth Law 同济大学 2010

The research process36

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Y d th k l d h i i t ti b

Trust

You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)

Kenneth Law 同济大学 2010

The research process37

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

(1) Some people have an in‐born tendency to share with

Trust

(1) Some people have an in‐born tendency to share with others

(2) Some people would expect reciprocal sharing in the future

(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)

(4) helliphellip

Kenneth Law 同济大学 2010

(4) helliphellip

Some questions38

1 Has anyone done anything on this topic before (literature review)

2 Which one is the most important reason to explain knowledge sharing (theory)

3 Are there established management theories that can explain the phenomenon (theory)

4 Are these antecedents or causes (theory)

5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)

6 What is the most appropriate research design to answer my research questionresearch question

Kenneth Law 同济大学 2010

When to use which design39

When to use which design

1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis

Kenneth Law 同济大学 2010

Do Not start your research40

Do Not start your research helliphellip

With some hypothesized relationships

With a model

Just because no one has done that before

Only because you see a significant correlation in your datasetyour dataset

Without looking at the literature

Without a research questionq

If you do not find it interesting

If you do not believe it is important

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

A different view33

A different view

With a research question you would automatically ask for a theoryautomatically ask for a theory

B d h ld Based on your theory you would automatically develop some hypotheses

Kenneth Law 同济大学 2010

The research process34

The research process

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Trusttendency behaviors

Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g

Kenneth Law 同济大学 2010

The research process35

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

H d k th t k l d h i i t ti i

Trust

How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g

This is because helliphellip (a theory)

Kenneth Law 同济大学 2010

The research process36

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Y d th k l d h i i t ti b

Trust

You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)

Kenneth Law 同济大学 2010

The research process37

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

(1) Some people have an in‐born tendency to share with

Trust

(1) Some people have an in‐born tendency to share with others

(2) Some people would expect reciprocal sharing in the future

(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)

(4) helliphellip

Kenneth Law 同济大学 2010

(4) helliphellip

Some questions38

1 Has anyone done anything on this topic before (literature review)

2 Which one is the most important reason to explain knowledge sharing (theory)

3 Are there established management theories that can explain the phenomenon (theory)

4 Are these antecedents or causes (theory)

5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)

6 What is the most appropriate research design to answer my research questionresearch question

Kenneth Law 同济大学 2010

When to use which design39

When to use which design

1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis

Kenneth Law 同济大学 2010

Do Not start your research40

Do Not start your research helliphellip

With some hypothesized relationships

With a model

Just because no one has done that before

Only because you see a significant correlation in your datasetyour dataset

Without looking at the literature

Without a research questionq

If you do not find it interesting

If you do not believe it is important

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

The research process34

The research process

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Trusttendency behaviors

Are you interested in knowledge sharing intention or knowledge sharing behaviorsg g

Kenneth Law 同济大学 2010

The research process35

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

H d k th t k l d h i i t ti i

Trust

How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g

This is because helliphellip (a theory)

Kenneth Law 同济大学 2010

The research process36

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Y d th k l d h i i t ti b

Trust

You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)

Kenneth Law 同济大学 2010

The research process37

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

(1) Some people have an in‐born tendency to share with

Trust

(1) Some people have an in‐born tendency to share with others

(2) Some people would expect reciprocal sharing in the future

(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)

(4) helliphellip

Kenneth Law 同济大学 2010

(4) helliphellip

Some questions38

1 Has anyone done anything on this topic before (literature review)

2 Which one is the most important reason to explain knowledge sharing (theory)

3 Are there established management theories that can explain the phenomenon (theory)

4 Are these antecedents or causes (theory)

5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)

6 What is the most appropriate research design to answer my research questionresearch question

Kenneth Law 同济大学 2010

When to use which design39

When to use which design

1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis

Kenneth Law 同济大学 2010

Do Not start your research40

Do Not start your research helliphellip

With some hypothesized relationships

With a model

Just because no one has done that before

Only because you see a significant correlation in your datasetyour dataset

Without looking at the literature

Without a research questionq

If you do not find it interesting

If you do not believe it is important

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

The research process35

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

H d k th t k l d h i i t ti i

Trust

How do you know that knowledge sharing intention is affected by benevolence expected reciprocity and interpersonal trust Why these three Why three only Why not emotions at the time of sharing y g

This is because helliphellip (a theory)

Kenneth Law 同济大学 2010

The research process36

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Y d th k l d h i i t ti b

Trust

You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)

Kenneth Law 同济大学 2010

The research process37

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

(1) Some people have an in‐born tendency to share with

Trust

(1) Some people have an in‐born tendency to share with others

(2) Some people would expect reciprocal sharing in the future

(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)

(4) helliphellip

Kenneth Law 同济大学 2010

(4) helliphellip

Some questions38

1 Has anyone done anything on this topic before (literature review)

2 Which one is the most important reason to explain knowledge sharing (theory)

3 Are there established management theories that can explain the phenomenon (theory)

4 Are these antecedents or causes (theory)

5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)

6 What is the most appropriate research design to answer my research questionresearch question

Kenneth Law 同济大学 2010

When to use which design39

When to use which design

1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis

Kenneth Law 同济大学 2010

Do Not start your research40

Do Not start your research helliphellip

With some hypothesized relationships

With a model

Just because no one has done that before

Only because you see a significant correlation in your datasetyour dataset

Without looking at the literature

Without a research questionq

If you do not find it interesting

If you do not believe it is important

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

The research process36

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

Y d th k l d h i i t ti b

Trust

You need a theory on knowledge sharing intention because you can start researching this variable Why would people share knowledge with other (I do not need variables I need an explanation a perspective a theory)p p p y)

Kenneth Law 同济大学 2010

The research process37

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

(1) Some people have an in‐born tendency to share with

Trust

(1) Some people have an in‐born tendency to share with others

(2) Some people would expect reciprocal sharing in the future

(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)

(4) helliphellip

Kenneth Law 同济大学 2010

(4) helliphellip

Some questions38

1 Has anyone done anything on this topic before (literature review)

2 Which one is the most important reason to explain knowledge sharing (theory)

3 Are there established management theories that can explain the phenomenon (theory)

4 Are these antecedents or causes (theory)

5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)

6 What is the most appropriate research design to answer my research questionresearch question

Kenneth Law 同济大学 2010

When to use which design39

When to use which design

1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis

Kenneth Law 同济大学 2010

Do Not start your research40

Do Not start your research helliphellip

With some hypothesized relationships

With a model

Just because no one has done that before

Only because you see a significant correlation in your datasetyour dataset

Without looking at the literature

Without a research questionq

If you do not find it interesting

If you do not believe it is important

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

The research process37

The research process

Benevolence

Expected returnknowledge sharing tendency

knowledge sharing

behaviors

Benevolence

(1) Some people have an in‐born tendency to share with

Trust

(1) Some people have an in‐born tendency to share with others

(2) Some people would expect reciprocal sharing in the future

(3) Some people use knowledge sharing as an instrumental means (ie to get something they want at the end)

(4) helliphellip

Kenneth Law 同济大学 2010

(4) helliphellip

Some questions38

1 Has anyone done anything on this topic before (literature review)

2 Which one is the most important reason to explain knowledge sharing (theory)

3 Are there established management theories that can explain the phenomenon (theory)

4 Are these antecedents or causes (theory)

5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)

6 What is the most appropriate research design to answer my research questionresearch question

Kenneth Law 同济大学 2010

When to use which design39

When to use which design

1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis

Kenneth Law 同济大学 2010

Do Not start your research40

Do Not start your research helliphellip

With some hypothesized relationships

With a model

Just because no one has done that before

Only because you see a significant correlation in your datasetyour dataset

Without looking at the literature

Without a research questionq

If you do not find it interesting

If you do not believe it is important

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Some questions38

1 Has anyone done anything on this topic before (literature review)

2 Which one is the most important reason to explain knowledge sharing (theory)

3 Are there established management theories that can explain the phenomenon (theory)

4 Are these antecedents or causes (theory)

5 Do you want to study all core antecedents at the same time 5 Do you want to study all core antecedents at the same time (coverage)

6 What is the most appropriate research design to answer my research questionresearch question

Kenneth Law 同济大学 2010

When to use which design39

When to use which design

1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis

Kenneth Law 同济大学 2010

Do Not start your research40

Do Not start your research helliphellip

With some hypothesized relationships

With a model

Just because no one has done that before

Only because you see a significant correlation in your datasetyour dataset

Without looking at the literature

Without a research questionq

If you do not find it interesting

If you do not believe it is important

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

When to use which design39

When to use which design

1 Qualitative reviews2 Qualitative studies3 E i l l b di3 Experimental laboratory studies4 Quasi experiments5 Correlational survey studies5 Correlational survey studies6 Meta analysis

Kenneth Law 同济大学 2010

Do Not start your research40

Do Not start your research helliphellip

With some hypothesized relationships

With a model

Just because no one has done that before

Only because you see a significant correlation in your datasetyour dataset

Without looking at the literature

Without a research questionq

If you do not find it interesting

If you do not believe it is important

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Do Not start your research40

Do Not start your research helliphellip

With some hypothesized relationships

With a model

Just because no one has done that before

Only because you see a significant correlation in your datasetyour dataset

Without looking at the literature

Without a research questionq

If you do not find it interesting

If you do not believe it is important

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

A suggested research procedure hellip41

A suggested research procedure hellip

Starts with a research questionq

Think about from what perspective you would dd thi iaddress this issue

Develop testable hypotheses based on your Develop testable hypotheses based on your perspective to solve the research question

Wh i h b h d h h h What is the best method to test these hypotheses

Research design

Kenneth Law 同济大学 2010

Research design helliphellip

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Issues in survey design 42

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

2 What are the hypotheses43

2 What are the hypotheses

A th th ti l di f th h th 1 Are there theoretical groundings for the hypothesesPast resultsLogicLogicTheory

2 Are the hypotheses adding to the current literature yp grelating to the key construct

3 Are the hypotheses tautological4 Are the hypotheses too obvious to be true

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Three possible arguments44

Three possible arguments

1 Bass and Bentler (2001) found that followers who1 Bass and Bentler (2001) found that followers who followed transformational leaders have stronger vision of where the firm is heading to As a result we hypothesize that helliphellipyp

2 A transformational leader leads by creating visions for hisher followers They share their visions with their followers and communicate with their followers continuous on these visions Since mission and vision is a core component of organizational commitment we hypothesize thathelliphellip

3 According to the social exchange theory leader-follower relationship who engage in social exchange would expect long term reciprocity instead of immediate

d th f h th i d th t

Kenneth Law 同济大学 2010

reward we therefore hypothesized that helliphellip

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Issues in survey design 45

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

3 Measure your construct of interest46

3 Measure your construct of interest

a) What is your level of analysisb) What is your data sourcec) Use validated scales if possibled) The scale development processe) Formative vs Reflective indicators

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Issues in survey design 47

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

3 l f A l i

48

3a Level of Analysis Individual levelgroup levelfirm levelindustry Individual levelgroup levelfirm levelindustry

levelcross levelExample Example 1 The effects of LMX on employee performance2 On the antecedents and outcomes of group‐2 On the antecedents and outcomes of group

level OCB3 The effect of HRM practices on firm

performance4 The effect of HRM practices on the job

satisfaction of employees

Kenneth Law 同济大学 2010

satisfaction of employees

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Additive model49

Additive model In additive composition models the meaning of the

hi h l l t t i ti f th l l l ithigher level construct is a summation of the lower level unitsregardless of the variance among these units In additive composition models the variance of the lower level units is of

th ti l ti l f i th lno theoretical or operational concern for composing the lower level construct to the higher level construct The typical operational combination process is a simple sum or average of th l l l th l l l i bl t tthe lower level scores on the lower level variable to represent the value on the higher level variable

Eg average organizational commitment of employees in an organization

Eg total productivity of employees in a firm

Kenneth Law 同济大学 2010

Reference Chan David (1998) Functional Relations among constructs in the same content domain at different levels of analysisrdquo A typology of composition models Journal of Applied Psychology 83(2) 234-246

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Additive model50

Additive model

Productivity

Employee 1 + Employee 2 + hellip

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Direct consensus model51

Direct consensus model Direct consensus composition uses within-group consensus of

th l l l it th f ti l l ti hi t ifthe lower level units as the functional relationship to specify how the construct operationalized at the lower level is functionally isomorphic to another form of the construct at the hi h l l Th t i l ti l bi ti ihigher level The typical operational combination process is using within-group agreement of scores to index consensus at the lower level and to justify aggregation of lower level scores t t t th hi h l lto represent scores at the higher level

Eg use within group agreement of scores to index consensus at the lower level about the justice perceptions of individual j p pemployees and to justify aggregation of lower level scores to represent scores at the higher level

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Direct consensus model52

Direct consensus modelLeader characteristics

Group 1 outputsEmployee 1 group 1Employee 2 group 1 X1p y g pEmployee 3 group 1

Group 2 outputsEmployee 1 group 2Employee 2 group 2Employee 3 group 2

X2Employee 3 group 2

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Reference shift model53

Reference shift modelbull The critical difference between reference shift consensus and

direct consensus of composition is that in referent shiftdirect consensus of composition is that in referent-shift consensus composition the lower level attributes being assessed for consensus are conceptually distinct though derived from the original individual-level construct That is g there is a shift in the referent prior to consensus assessment and it is the new referent that is actually being combined to represent the higher level construct

bull If we use organizational climate as an example rather than an individualrsquos own climate perceptions (ie psychological climate) or the aggregation of individuals perceptions (i eclimate) or the aggregation of individuals perceptions (ie organizational climate) the researcher now is interested in how an individual believes others in the organization perceive the climate and whether there is within-organization consensus

Kenneth Law 同济大学 2010

gin such beliefs

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Reference shift model54

Reference shift modelFirm citizenship behaviors

Firm 1 turnoverManager 1 group 1Manager 2 group 1 X1g g pManager 3 group 1

Firm 2 turnoverManager 1 group 2Manager 2 group 2Manager 3 group 2

X2Manager 3 group 2

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Three examples55

Three examples

Addi i d l1 Additive model我很樂意幫助組內的同事

2 Direct consensus model我認為這是個合作的小組

3 Reference‐shift model我的小組成員都認為這是個合作的小組我的小組成員都認為這是個合作的小組

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Issues in survey design 56

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

3b Data source and CMV57

3b Data source and CMV

Try to solicit data (esp predictor vs criterion variables) from different )sources

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

An example58

An example

HRM practices of

th fi

Degree of social exchange in the

i ti

Individual performance f lthe firm organization of employees

Source of informationSource of information

HR manager Middle managers Top level managersg g p g

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Common Method Variance59

Common Method Variance

The problem of common method variance refers to the presence ofvariance refers to the presence ofspurious correlation between two variable which is caused by avariable which is caused by acommon third variable when they

d b h h dare measured by the same method

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

An exampleAn example

I am committed to this organization

I am very satisfied with my current job

Organizational commitment

Job satisfaction

Self-report Self-report

Negative(positive) affectivity

The respondent is a critical judgmental person60

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Another common third variablesAnother common third variables

A person with high social desirability is one who p g yhas a strong inclination to present himselfherself positively to othersOganizational commitment (affective)1我很乐意在此家公司中渡过我余下的生涯

2 这家公司所面临的问题就是我自己的问题2这家公司所面临的问题就是我自己的问题

3我有很强地属于「这家公司的人」的感觉

Turnover Intention7 我常想到辞职

8 我很可能在明年另寻新的工作

9 如果能自由选择 我仍然喜欢留在这机构工作9 如果能自由选择我仍然喜欢留在这机构工作

61

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

IllustrationIllustration

I am committed to this organization

I often think about leaving this

organizationOrganizational commitment

Turnover Intention

organization

Self-report Self-report

Social Desirability

The respondent likes to give a good impression to others62

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

A Method Factor63

A Method Factor

Oganizational commitment (affective) 不同意 同意

1我很乐意在此家公司中渡过我余下的生涯 1 2 3 4 5

2这家公司所面临的问题就是我自己的问题 1 2 3 4 5

3我有很强地属于「这家公司的人」的感觉 1 2 3 4 5

Turnover IntentionTurnover Intention7 我很少想到辞职 1 2 3 4 5

8 我不可能在明年另寻新的工作 1 2 3 4 5

9 如果能自由选择我仍然喜欢留在这机构工作 1 2 3 4 5

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

64

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Partialling Venn diagramPartialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

65

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Rotated Factor Matrix in EFA66

Factors

Rotated Factor Matrix in EFA

FactorsVar A B C DX1 29 60 -06 02X 32 81 12 - 03

Organizational commitment X2 32 81 12 03

X3 35 77 03 08X4 27 01 65 -04X5 41 03 80 07

commitment

Job satisfaction5

X6 40 12 67 -05X7 33 19 -02 68X8 22 08 -10 53

Turnover intention

X9 45 26 -13 47intention

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Partial out by a method factor67

Partial out by a method factor

Organizational commitment

Turnover intention

Items 1 2 3 4 5 6 7 8 9

Negative affectivity

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Partialling --Venn diagram

Organizational commitment

Turnover intention

Negativegaffectivity

68

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Include in the model69

Include in the model

Organizational commitment

Job satisfaction

Negativeaffectivity

Other missing method variables

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Different methodssource70

Different methodssource

Organizational commitment

Organizational culture commitmentculture

bull Not reported by employee bull Self reported by employeebull Not reported by employeebull rites and ceremonials

bull Self reported by employee

reported by employee reported by supervisorpeer

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Issues in survey design 71

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

3c What measure to be used72

3c What measure to be used Use full scale of existing validated scales Select items only when you have perfect justificationsy y p j Use scales that have been validated (esp cross culturally) Develop you own measure when you have strong reason that existing

measures do not fit or there is no good measure of the construct

MeasureMeasureEmotional intelligence was measured by five items adapted from Law Wong and Song (2004) One sample item is ldquoI am able to control my temper most of the timerdquo y pCoefficient of the five items was 89

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Using existing scales73

Using existing scales

MeasureWe developed five items to measure emotional intelligence in this study One sample item is ldquoI am able to control my temper most of the timerdquo Coefficient of the five items was 89

Problems1 We do not know how the items are developed2 There is no evidence of validity of the items2 There is no evidence of validity of the items3 We do not know whether you have done any item

trimming or not4 If yes we do not know the criteria of item selection

Kenneth Law 同济大学 2010

4 If yes we do not know the criteria of item selection

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Using existing scales74

Using existing scales

bull Follow the proper procedure of scaleFollow the proper procedure of scale translation

bull The minimum requirement is a forward-qbackward translation

bull It is best to pre-test your (translated) scale b fbefore use

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Issues in survey design 75

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

3d Developing new scales76

3d Developing new scales

Inductive vs deductive approach for scale Inductive vs deductive approach for scale development

Inductivebull Usually behavioral measures of constructsbull Eg Managers write statements to describeEg Managers write statements to describe

behaviors of a transformational leaderbull Researcher group all items and sort them into

various dimensions using systematic classificationvarious dimensions using systematic classification techniques

bull Select items to represent each dimension

Kenneth Law 同济大学 2010

bull Pretesting of the scale

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Developing new scales77

Developing new scalesDeductivebull Start with theory to determine the dimensionality

of the constructbull For each and every dimension draft items to y

represent the dimensionbull Pretesting of the scalebull Item trimmingbull Item trimmingbull Final validation

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Pros amp Cons78

Pros amp Cons

InductiveInductive items derived from the respondentsrsquo perspectives items may frequently capture variances outside the y q y p

domain of the construct (eg transformational leadership including items which measure whether the leader is hardworking)g)

Who is the person to decide which item to includeDeductive Good theoretical basis Ensure Content validity May not be respondent friendly

Kenneth Law 同济大学 2010

May not be respondent friendly

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Scale development process79

Scale development processStep 1 Item Generation (inductive vs deductive)

Step 2 Questionnaire Administration

Step 3 Initial Item Reduction (EFA)

Step 4 Confirmatory Factor Analysis

Step 5 ConvergentDiscriminant ValidityStep 5 ConvergentDiscriminant Validity

Step 6 Replication

Kenneth Law 同济大学 2010

Adapted from Hinkin TR (1998) A brief tutorial on the development of measures for use in survey questionnaires Organizational Research Methods 1 104-121

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Issues in scale development80

Issues in scale development

Precision vs reliability (e g one item for measuring Precision vs reliability (eg one item for measuring organizational justice 用人唯賢 使用道德操手高的下屬使用道德操手高的下屬 使用能作眾人榜樣的下屬 使用樂於幫助別人的下屬

Leading questions usually define the frame‐of‐reference Leading questions usually define the frame of reference of the respondent

Anchoring (4 5 6 7) Negatively worded items (e g 在上班時間做私人事情) Negatively worded items (eg 在上班時間做私人事情)

Behaviorsattitudes (eg in measuring leadership‐他是一個負責任的領導

他會悉心教導下屬

Kenneth Law 同济大学 2010

‐他會悉心教導下屬

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Issues in scale development81

Issues in scale development

Item selection (EFA IRT)

IRT (item difficulty and item reliability)( y y)

Item difficulty ndash what is the proportion of respondents who will give high ratings on this item

Item reliability ndash the correlation of an item score with the total scores measuring the construct (coefficient )

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Item Response Theory82

Let us use a simply yes (1) or no (0) question to illustration

P ti l f d tProportional of respondents choosing ldquoyesrdquo for this item

B

C

A

B

A

D

Kenneth Law 同济大学 2010

Total score of that construct

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Distribution of total score83

Frequency

хх х

х

х

х

Total score of all items1 2 3 4 5 6

х

Kenneth Law 同济大学 2010

Total score of all items measuring this construct

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Issues in survey design 84

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

3e Formative vs Reflective indicators85

Income Relax

Socio-economic

statusParentrsquos income

Life satisfaction Happy

Size of apartment Positive

Formative or causal indicators Reflective or effect indicators

Please give one example of each type of construct

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Measurement models86

Measurement models

For formative indicators

Indicators are purely theory driven

O d t d hi h ffi i t One does not need high coefficient One does not need to conduct EFACFA

Content coverage of indicators is crucial Content coverage of indicators is crucial

Measurement model identification is an important issue

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Issues in survey design 87

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Multidimensional constructs88

Multidimensional constructs

Quality

Job Performance

y

Quantity

On time

Aggregate Model

On‐time

Math

Mental Ability Verbal

Memory

Latent Model

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Issues in survey design 89

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

4 Pilot Test90

4 Pilot Test

Item trimming (EFA) Factor loading gt4 Low cross loading Low cross loading Item difficultyItem reliability Never trim items based on EFA and then retest with a

CFA using the same sample Cross validation

C t amp di i i t lidit Convergent amp discriminant validity

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Confirmatory Factor Analysis 91

FactorsVar F1 F2 F3 h2

x1 60 -06 02 36x2 81 12 - 03 67

Exploratory factor analysis

Factors

x2 81 12 03 67x3 77 03 08 60x4 01 65 -04 42x5 03 80 07 65x6 12 67 -05 47

19 02 68 0

Does the data ldquofitrdquo our specified model

FactorsVar F1 F2 F3

x1 60 0 081 0 0Guanxi

x7 19 -02 68 50x8 08 -10 53 30x9 26 -13 47 31

x2 81 0 0x3 77 0 0x4 0 65 0x 0 80 0

Guanxi

promotion x5 0 80 0x6 0 67 0x7 0 0 68x8 0 0 53

promotion

Bonus

Kenneth Law 同济大学 2010

x8 0 0 53x9 0 0 47

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Fit Indices in CFA92

Fit Indices in CFA

Model 2 and d f Model and df CFI gt 90TLI (NNFI) TLI (NNFI) gt 90

SRMR lt 05 RMSEA lt 05

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Issues in survey design 93

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

5 Convergent amp Discriminant Validity94

5 Convergent amp Discriminant ValidityConvergent Validityg y Two independent methods of inferring an attribute lead

to similar ends (Nunnally amp Bernstein 1994 p92)Oft i l l ti ith Often involves correlating a new measure with an existing measure

DiscriminantValidityy Measures of different attributes should not correlate to

an extremely high degreeOf i l l i i h l d Often involves correlating a new measure with related measures

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

A sample MTMM matrix95

A sample MTMM matrix(Paper amp Pencil self test)

Heterotrait-monomethod

MonotraitMonotrait-monomethodMonotrait-

heteromethod

Heterotrait-heteromethod

Kenneth Law 同济大学 2010

Adapted from httpwwwsocialresearchmethodsnetkbmtmmmathtm

Note SE self esteem SD self disclosure LC Locus of control

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Elements of a MTMM matrix96

Elements of a MTMM matrix

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

MTMM for construct validity97

f ySelf-rating Parent Rating

Traits EI NEU

EXT OPEN ANT

CON EI NEU

EXT OPEN ANT

CON

EI (78)NEU -39 (77)

Self-rating

( )EXT 15 -08 (80)OPEN 30 -12 45 (82)ANT 26 -36 29 14 (83)CON 55 -46 10 27 47 (86)EI 28 -12 00 01 02 22 (81)NEU -18 34 04 -02 -18 -20 -30 (79)

Parent Rating

EXT 06 -02 37 21 02 -02 00 08 (83)OPEN 15 -04 14 32 -10 08 15 08 55 (85)ANT 07 -14 01 -02 20 14 16 -16 28 09 (85)CON 17 11 13 02 05 34 42 21 11 24 58 ( 90)

Kenneth Law 同济大学 2010

CON 17 -11 -13 -02 05 34 42 -21 11 24 58 (90)

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Interpreting MTMM98

Interpreting MTMM

R li bilit ( t it th d) h ld b th Reliability (monotrait‐monomethod) should be the highest

Monotrait‐heteromethod (convergent validity) must be o ot a t ete o et od (co e ge t a d ty) ust begt0 and high

Monotrait‐heteromethod gt heterotrait‐monomethod d l d h h h d(discriminant validity)gt heterotrait‐heteromethod

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

SEM analysis of MTMM99

SEM analysis of MTMM

Trait 1 Trait 2

T1M1 T1M2 T2M1 T2M2

Method 1 Method 2Method 1 Method 2

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Issues in survey design 100

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

6 Questionnaire Design101

Q g1 Question sequencing

bull Dependent variables firstDependent variables firstbull Randomization

2 Grouping of constructs3 Number of response categories (4 5 6 7)3 Number of response categories (4567)4 Length of questionnaire ( of pages)5 What constructs to include (two papers but not too long)

第三部分下面这些陈述是有关您自己对工作及医院的一些想法对于每一题目请在后面最能代表您的意见的选项上划圈表您的意见的选项上划圈

1在生活中看重的事和我单位看重的事很相似 bull1 bull2 bull3 bull4 bull5

2我个人的价值观和我单位的价值观及文化相符 bull1 bull2 bull3 bull4 bull5

Kenneth Law 同济大学 2010

3我单位的价值观及文化和我在生活中看重的相符 bull1 bull2 bull3 bull4 bull5

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Issues in survey design 102

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

7 Data collection103

7 Data collection

1 Minimum N is 15 (one respondent for each item within a construct))

2 Minimum N gt100 for group level gt200 for individual level

3 You should be there during data collection

4 Questionnaire distribution ndash the higher the level the betterbetter

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Issues in survey design 104

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

8 Data analysis105

8 Data analysis

1 Confirmatory factor analysis of all items from the 1 Confirmatory factor analysis of all items from the same source

2 Separate measurement model from structural modelU f t 3 Use mean score or factor score

4 Using parcels when number of items are large5 Saving your SPSSSASSTATA program file g y p g6 Never trim items based on EFA and then retest with

a CFA using the same sample7 Control for social desirability or affectivity7 Control for social desirability or affectivity

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Issues in survey design 106

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

8aForming parcels in CFAg p

3)( 3211 xxxg

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

2 4 5 6( ) 3g x x x

3 7 8( ) 2g x x

107

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Issues in survey design 108

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

8b Cross level analysis8bCross level analysis

1 Different sources of data1 Different sources of data2 One supervisor rating several

subordinates (autocorrelation)subordinates (autocorrelation)3 Rwg amp ICC4 HLM4 HLM

109

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Individual-group analysis110

Individual group analysis

Subordinates nested within supervisorsp Aggregate of individual data to group level Justification of analysis at the individual level

Leadership PerformancepSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

=333 =267

Sup 2 Sub 1 3 2Sub 2 2 5Sub 3 5 3Sub 4 2 2

=300 =300

Kenneth Law 同济大学 2010

Sub 4 2 2

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Cross level analysisy1 One supervisor rating several subordiantes

( t l ti )(autocorrelation)2 Rwg amp ICC3 Hierachical Linear Modeling

Job Sat PerformanceSup 1 Sub 1 5 3

Sub 2 2 1Sub 3 3 4

Sup 2 Sub 1 3 2Sub 2 2 5S b 3 3Sub 3 5 3Sub 4 2 2

111

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Hierarchical Linear Modelingg

( )Level Helping b b Mood r 1 0j 1j

2 0j 00 01 0j

( )

(Pr )ij ij ij

j

Level Helping b b Mood r

Level b oximity u

where = Level‐2 intercept

1j 10 1j b u

00 = Level 2 intercept01 = Level‐2 slope 0 = mean (pooled) slopes across groupVariance (r ) = 2 = Level‐1 residual variance Variance (rij) = = Level‐1 residual variance Variance (u0j) = 00 = residual intercept varianceVariance (u1j) = 11 = variance in slopes

112

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Hierarchical Linear Modeling113

Hierarchical Linear Modeling

t l l ti b d b HLM1 not every cross level question can be answered by HLM

2 HLM does not consider measurement model

3 There can only be one dependent variable in HLM3 There can only be one dependent variable in HLM

4 Only group level variables affect individual level intercept and slope in HLM

5 The exact procedure of testing mediators using HLM is not established yet

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Issues in survey design 114

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

8 M d i d M di i

115

8cModeration and Mediation

Moderator Interaction effectsMediatorMediator

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Moderator116

Moderator

A moderator variable is any variable which when ysystematically varied ldquocausesrdquo the relationship between two other variables to change(Stone p 26)(Stone p26)

Moderator variable is a variable whose different values d t i th t f th l ti hi b t tdetermine the nature of the relationship between two other variables (Schmitt amp Klimoski p87)

Kenneth Law 同济大学 2010

Stone E (1978) Research Methods in Organizational Behavior DallasScott Foresman amp CompanySchmitt NW amp Klimoski R (1991) Research Methods in Human Resources Management Cincinnati South-Western Publishing Co

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Moderation and Interaction117

EmotionalLabor

EI Performance

Labor

EI

PerformanceThe total effects is greater than the

Mental Ability

summed effects

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

MediatorsMediators

Mediators are variables that mediate theMediators are variables that mediate the relationship between two other variables The mediating variable is caused by one variable but it in turn causes a third variable (Schmitt amp Klimoski p87)

A mediator is a variable which two variables

118

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Mediated Regression119

g

OCB OCB OCB OBSEFull or partial mediation

OCB OCB OCB OBSESimilarity (X) 17 28 11 35OBSE (M) 23Adj t d R2 10 36Adjusted R2 10 36

R2

Similarity

OBSE

OCB

Note1 Mediator (Baron amp Kenny 1986) approach2 The importance of ca salit

Kenneth Law 同济大学 2010

2 The importance of causality

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Mediating and indirect effects120

Mediating and indirect effects

J b A i tJob Assignment

f

LMX

29

19

44

Performance ratings

Guanxi

29

2983 5271

Bonus allocation

Commitmentto supervisor 21

Chances of promotion

p

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

Issues in survey design 121

1 What is the research question

2 What are the hypotheses

3 Measure your construct of interest3 Measure your construct of interesta) What is your level of analysisb) What is your data sourcec) Use validated scales if possible) pd) The scale development processe) Formative vs Reflective indicatorsf) Multidimensional constructs

4 Pilot test

5 Convergent amp Discriminant Validity

6 Issues in Questionnaire design (2) Explanation

(3) Model amp Theory(4) prediction (5) Application

Y

X2

X1

6 Issues in Questionnaire design

7 How to collect data

8 Data analysis(1) Observation

(2) Explanation

H1H2helliphellip

Kenneth Law 同济大学 2010

a) Confirmatory Factor Analysisb) Need cross level analysisc) Mediators and moderators

Theory Building Theory Testing Theory Application

122

E dEnd

Kenneth Law 同济大学 2010

122

E dEnd

Kenneth Law 同济大学 2010