validity and reliability of questionnaires
TRANSCRIPT
-
8/19/2019 Validity and Reliability of Questionnaires
1/38
UNIT 3 CONTD…..
VALIDITYAND
RELIABILITY OFQUESTIONNAIRES
BY – JASKARAN SINGH
JASKARAN SINGH
-
8/19/2019 Validity and Reliability of Questionnaires
2/38
CONTENTS
Introduction
Steps in questionnaire designing
Validity Concept o !alidity
"ypes o !alidity
Steps in questionnaire !alidation
Relia#ility "ypes and $easure$ent o relia#ility
Conclusion
JASKARAN SINGH
-
8/19/2019 Validity and Reliability of Questionnaires
3/38
INTRODUCTION
%uestionnaire& I$portant $et'od o datacollection used e(tensi!ely
Ad!antages o questionnaire )ess e(pensi!e
*+ers greater anony$ity
,isad!antages Application is li$ited
Response rate is lo-
*pportunities to clariy issues is lac.ing
JASKARAN SINGH
-
8/19/2019 Validity and Reliability of Questionnaires
4/38
Ideal requisites o a questionnaire& S'ould #e clear and easy to understand )ayout is easy to read and pleasant to eye
Sequence o questions easy to ollo- S'ould #e de!eloped in an interacti!e style Sensiti!e questions $ust #e -orded
e(actly
JASKARAN SINGH
-
8/19/2019 Validity and Reliability of Questionnaires
5/38
Steps in questionnaie!esi"nin"
JASKARAN SINGH
-
8/19/2019 Validity and Reliability of Questionnaires
6/38
Va#i!it$
JASKARAN SINGH
-
8/19/2019 Validity and Reliability of Questionnaires
7/38
T%e &on&ept o' (a#i!it$
Validity is t'e a#ility o an instru$ent to $easure -'at it isintended to $easure/
,egree to -'ic' t'e researc'er 'as $easured -'at 'e 'as
set out to $easure 0S$it'1 23324
Are -e $easuring -'at -e t'in. -e are $easuring50Kerlinger1 23674
8(tent to -'ic' an e$pirical $easure adequately re9ectst'e real $eaning o t'e concept under consideration0Ba##ie1 23:34
JASKARAN SINGH
-
8/19/2019 Validity and Reliability of Questionnaires
8/38
;'y !alidity 5
Validity is done $ainly to ans-er t'e ollo-ingquestions&
Is t'e researc' in!estigation pro!iding ans-ers to t'eresearc' questions or -'ic' it -as underta.en5
I so1 is it pro!iding t'ese ans-ers using appropriate$et'ods and procedures5
JASKARAN SINGH
-
8/19/2019 Validity and Reliability of Questionnaires
9/38
%uestions to ponder0"'in.a#out4
In!estigatorReaders oreport
8(perts in t'e
-
8/19/2019 Validity and Reliability of Questionnaires
10/38
Lo"i&a# t%in)in"
Justi
-
8/19/2019 Validity and Reliability of Questionnaires
11/38
Statisti&a# po&e!ues
By calculating coe>cient o correlations#et-een questions and outco$e!aria#les
JASKARAN SINGH
-
8/19/2019 Validity and Reliability of Questionnaires
12/38
T$pes o' (a#i!it$
Validity
Content!alidity
@ace !alidity
Criterionrelated
Concurrent redicti!e
Construct!alidity
JASKARAN SINGH
-
8/19/2019 Validity and Reliability of Questionnaires
13/38
CONTENT VALIDITY
ses logical reasoning and 'ence easy to apply
8(tent to -'ic' a $easuring instru$ent co!ers a
representati!e sa$ple o t'e do$ain o t'easpects $easured
;'et'er ite$s and questions co!er t'e ull range
o t'e issues or pro#le$ #eing $easured8(a$ple1 i -e -ant to test .no-ledge on
A$erican Geograp'y it is not air to 'a!e $ostquestions li$ited to t'e geograp'y o 8ngland/
JASKARAN SINGH
-
8/19/2019 Validity and Reliability of Questionnaires
14/38
@ace Validity
"'e e(tent to -'ic' a $easuring instru$entappears !alid on its surace
8ac' question or ite$ on t'e researc' instru$ent$ust 'a!e a logical lin. -it' t'e o#=ecti!e
JASKARAN SINGH
-
8/19/2019 Validity and Reliability of Questionnaires
15/38
CRITERION VALIDITY
"'e e(tent to -'ic' a $easuring instru$entaccurately predicts #e'a!iour or a#ility in a gi!enarea/
"'e $easuring instru$ent is called criteria?
It is o t-o types&
redicti!e !alidity Concurrent !alidity
JASKARAN SINGH
-
8/19/2019 Validity and Reliability of Questionnaires
16/38
redicti!e !alidity
I t'e test is used to predict uture peror$ance
8g& 8ntrance e(a$ / / / / eror$ance o t'ese
tests correlates -it' later peror$ance inproessional college
8g& ;ritten dri!ing test
JASKARAN SINGH
-
8/19/2019 Validity and Reliability of Questionnaires
17/38
Concurrent !alidity
I t'e test is used to esti$ate presentperor$ance or person?s a#ility at t'e presentti$e not atte$pting to predict uture outco$es
roessional college e(a$
8g& dri!ing test1 pilot test
JASKARAN SINGH
-
8/19/2019 Validity and Reliability of Questionnaires
18/38
CONSTRUCT VALIDITY
Dost i$portant type o !alidity
Assesses t'e e(tent to -'ic' a $easuringinstru$ent accurately $easures a t'eoreticalconstruct it is designed to $easure
Deasured #y correlating peror$ance on t'e test-it' peror$ance on a test or -'ic' construct!alidity 'as #een deter$ined/
"'is can #e done e(peri$entally1 e/g/1 i -e -antto !alidate a $easure o an(iety/ ;e 'a!e a'ypot'esis t'at an(iety increases -'en su#=ectsare under t'e t'reat o an electric s'oc.1 t'ent'e t'reat o an electric s'oc. s'ould increase
an(iety scores/ JASKARAN SINGH
-
8/19/2019 Validity and Reliability of Questionnaires
19/38
Anot'er $et'od is to s'o- t'at scores o t'e ne-test di+ers across people -it' di+erent le!els ooutco$es #eing $easured/
JASKARAN SINGH
-
8/19/2019 Validity and Reliability of Questionnaires
20/38
Su**a$ o' Va#i!it$
CONTENT CRITERION CONSTRUCT
C*NCRR8N "
R8,IC"IV8
;'at it$easures
;'et'er t'etest co!ers a
representati!esa$ple o t'edo$ains to #e$easured
"'e a#ility ot'e test to
esti$atepresentperor$ance
"'e a#ility ot'e test to
predict utureperor$ance
"'e e(tent to-'ic' t'e
instru$ent$easures at'eoreticalconstruct
Ho- it isacco$plis'ed
As. e(perts toassess t'etest toesta#lis' t'att'e ite$s arerepresentati!eo t'eoutco$e
Correlateperor$anceon t'e test-it' aconcurrent#e'a!iour
Correlateperor$anceon t'e test-it' a#e'a!iour inuture
Correlateperor$anceon t'einstru$ent-it' aperor$anceon anesta#lis'edinstru$ent JASKARAN SINGH
-
8/19/2019 Validity and Reliability of Questionnaires
21/38
Steps inquestionn
aie
(a#i!ation JASKARAN SINGH
-
8/19/2019 Validity and Reliability of Questionnaires
22/38
FACE VALIDITY
8!aluate in ter$s o&
Reada#ility
)ayout andstyle
Clarity o-ording
@easi#ility
JASKARAN SINGH
-
8/19/2019 Validity and Reliability of Questionnaires
23/38
CONTENT VALIDITY
"-o p'ases
Researc'er& ConceptualiEationand do$ain analysis0
-
8/19/2019 Validity and Reliability of Questionnaires
24/38
+o, !o e-pets e(a#uate(a#i!it$ Det'od 2& A!erage Congruency ercentage
0AC4 Fop'a$1 236:
8(perts co$pute t'e percentage o questionsdee$ed to #e rele!ant or t'e$
"a.e t'e a!erage o all e(perts
I t'e !alue is 3 / / / Valid
8g& e(perts / / 08(pert 22L1 8(pert :L4
"'en AC M 3L
JASKARAN SINGH
-
8/19/2019 Validity and Reliability of Questionnaires
25/38
CONSTRUCT VALIDITY
"o e(a$ine e$pirically t'e interrelations'ip a$ong ite$s andto identiy clusters o ite$s t'at s'are su>cient !ariation to =ustiy t'eir e(istence as a actor or construct to #e $easured#y t'e instru$ent
Det'od& @actor analysis
Various ite$s are gat'ered into co$$on actors
Co$$on actors are synt'esiEed into e-er actors and t'enrelation #et-een eac' ite$ and actor is $easured
nrelated ite$s are eli$inated
JASKARAN SINGH
-
8/19/2019 Validity and Reliability of Questionnaires
26/38
Re#iai#it$
JASKARAN SINGH
-
8/19/2019 Validity and Reliability of Questionnaires
27/38
RELIABILITY
,e
-
8/19/2019 Validity and Reliability of Questionnaires
28/38
Relia#ility $easured inaspects o&
• , o n e t o e n s u r e t ' a t s a $ e r e s u l t s a r e o # t a i n e d - ' e n u s e d c o n s e c u t i ! e l y o r t - o o r $ o r e t i $ e s
• " e s t K r e t e s t $ e t ' o d i s u s e d
S"ABI)I"Y
• " o e n s u r e a l l s u # p a r t s o a i n s t r u $ e n t $ e a s u r e t ' e s a $ e c ' a r a c t e r i s t i c 0 H o $ o g e n e i t y 4
• S p l i t K ' a l $ e t ' o dIN"8RNA)
C*NSIS"8NC Y
• B s e d - ' e n t - o o # s e r ! e r s s t u d y a s i n g l e p ' e n o $ e n o n s i $ u l a t a n e o u s l y
• I n t e r K r a t e r r e l i a # i l i t y
8%IVA)8NC8
JASKARAN SINGH
T t R t t #i i#it
-
8/19/2019 Validity and Reliability of Questionnaires
29/38
Test/Retest e#iai#it$0'o stai#it$1
"est ad$inistered t-ice to t'e sa$e participant atdi+erent ti$es
sed or t'ings t'at are sta#le o!er ti$e
8asy and straig'tor-ard approac'
seul or questionnaires1 c'ec.list1 rating scales etc
,isad!antages ractice e+ect 0$ainly or tests4
"oo s'ort inter!als in #et-een 0e+ect o $e$ory4
So$e traits $ay c'ange -it' ti$e JASKARAN SINGH
-
8/19/2019 Validity and Reliability of Questionnaires
30/38
Statistical calculation
Ad$inistration o instru$ent to a sa$pleon t-o di+erent occasions
Scores co$pared and calculated #yusing correlation coe>cient or$ula0pearson4
JASKARAN SINGH
-
8/19/2019 Validity and Reliability of Questionnaires
31/38
Correlation coe>cient
Deasures t'e degree o relations'ip #et-een t-osets o scores
Can range ro$ 2 to O2
indicates a#sence o any relations'ips
Coe#ation &oe2&ient Sten"t% o'e#ations%ip
O /6 to 2/ Strong
O /7 to /P3 Doderate
O / to /3 None to -ea.
JASKARAN SINGH
S #it % # #i i#it
-
8/19/2019 Validity and Reliability of Questionnaires
32/38
Sp#it %a#(es e#iai#it$0%o*o"enit$1
8specially appropriate -'en t'e test is !ery long
Split t'e contents o t'e questionnaire into t-o equi!alent'al!esQ eit'er odde!en nu$#er or
-
8/19/2019 Validity and Reliability of Questionnaires
33/38
R? M r2Or
0r M coe>cient o split 'al1 R? Mcoe>cient o entire test4
JASKARAN SINGH
I t t #i i#it
-
8/19/2019 Validity and Reliability of Questionnaires
34/38
Inte/ate e#iai#it$0Equi(a#en&e1
InterRater Relia#ility reers to statistical $easure$entst'at deter$ine 'o- si$ilar t'e data collected #ydi+erent raters are/ A rater is so$eone -'o is scoringor $easuring a peror$ance1 #e'a!ior1 or s.ill in a
'u$an or ani$al/
sed -'en a single e!ent is $easured si$ultaneouslyand independently #y t-o or $ore trained o#ser!ers
R M Nu$#er o agree$ents
Nu$#er o agree$ents O Nu$#er o disagree$ents
JASKARAN SINGH
-
8/19/2019 Validity and Reliability of Questionnaires
35/38
Su$$ary o Relia#ility
TESTRETEST
SLIT +ALF INTERRATER
;'at it
$easures
Sta#ility o!er
ti$e
8qui!alency o
ite$s
Agree$ent
#et-een raters
Ho- it isacco$plis'e
d
Ad$inistert'e sa$e test
to t'e sa$epeople at t-odi+erentti$es
Correlateperor$ance or
a group opeople on t-oequi!alent'al!es o sa$etest
Ha!e $ultipleresearc'ers
$easure sa$einstru$ent anddeter$inepercentage oagree$ent#et-een t'e$
JASKARAN SINGH
-
8/19/2019 Validity and Reliability of Questionnaires
36/38
Conusion
Validated questionnaire It is one -'ic' 'as undergone a !alidation
procedure to s'o- t'at it accurately $easures-'at it ai$s to1 regardless o -'o responds1-'en t'ey respond1 and to -'o$ t'ey respondor -'en selad$inistered and -'ose relia#ility'as also #een e(a$ined t'ere#y&
Reducing #ias and a$#iguities
Better quality o data and credi#le inor$ation
JASKARAN SINGH
-
8/19/2019 Validity and Reliability of Questionnaires
37/38
In a nuts%e## . . . .
A questionnaire can be reliablebut invalid . . .
But a valid questionnaire isalways reliable . . /
JASKARAN SINGH
-
8/19/2019 Validity and Reliability of Questionnaires
38/38
T+AN4
YOU . .JASKARAN SINGH