measurement and scales validity & reliability error

Download Measurement and Scales Validity & Reliability Error

Post on 22-Dec-2015

214 views

Category:

Documents

2 download

Embed Size (px)

TRANSCRIPT

  • Slide 1
  • Measurement and Scales Validity & Reliability Error
  • Slide 2
  • Measurement
  • Slide 3
  • Measurement and Measurement Scales Measurement Process of assigning numbers or labels to things in accordance with specific rules to represent quantities or qualities of attributes. Rule: A guide, method, or command that tells a researcher what to do. Scale: A set of symbols or numbers constructed to be assigned by a rule to the individuals (or their behaviors or attitudes) to whom the scale is applied.
  • Slide 4
  • Types of Measurement Scales Nominal Scales Scales that partition data into mutually exclusive and collectively exhaustive categories. Ordinal Scales Nominal scales that can order data.
  • Slide 5
  • Interval Scales Ordinal scales with equal intervals between points to show relative amounts; may include an arbitrary zero point. Ratio Scales Interval scales with a meaningful zero point so that magnitudes can be compared arithmetically.
  • Slide 6
  • Nominal Ordinal Interval Ratio WinPlace Show 1 length2 lengths 40 to 1 long-shot pays $40
  • Slide 7
  • Type of ScaleNumerical OperationDescriptive Statistics NominalCountingFrequency; Percentage; mode OrdinalRank ordering(plus)Median Range; Percentile IntervalArithmetic operations on intervals bet numbers (plus) Mean; Standard deviation; variance RatioArithmetic operations on actual quantities (plus) Geometric mean; Co-efficent of variation
  • Slide 8
  • Selecting appropriate univariate statistical method ScaleBusiness Problem Statistical question to be asked Possible test of statistical significance Nominal ScaleIdentify sex of key executives Is the number of female executives equal to the number of males executives? Chi-square test
  • Slide 9
  • ScaleBusiness Problem Statistical question to be asked Possible test of statistical significance Nominal ScaleIndicate percentage of key executives who are male Is the proportion of male executives the same as the hypothesized proportion? T-test
  • Slide 10
  • ScaleBusiness Problem Statistical question to be asked Possible test of statistical significance Ordinal scaleCompare actual and expected evaluations Does the distribution of scores for a scale with categories of poor,good, excellent differ from an expected distribution? Chi-square test
  • Slide 11
  • ScaleBusiness Problem Statistical question to be asked Possible test of statistical significance Interval or Ratio scale Compare actual and hypothetical values of average salary Is the sample mean significantly different from the hypothesized population mean? Z-test (sample is large) T-test (sample is small)
  • Slide 12
  • 30/10/02 12 Error in Survey Research Random Sampling Error (Random error) Error that results from chance variation Impact can be decreased by increasing sample size and through statistical estimation (confidence interval) or rule of thumb Systematic Error (non sampling error) Error that results for the research design or execution.
  • Slide 13
  • Slide 14
  • 30/10/02 14 Types of Systematic Error 1. Administrative Error Error that results from improper execution. Data Processing Error Quality of data depends on quality of data entry. Use of verification procedures can minimize
  • Slide 15
  • 30/10/02 15 Sample Selection Error Systematic error resulting from improper sampling techniques either in design or execution. Interviewer Error Data recorded incorrectly (error or selective perception). Interviewer Cheating Mitigate by random checks
  • Slide 16
  • 30/10/02 16 2. Respondent Error Humans interviewing humans... Non-response error Statistical difference between a survey that includes only those who responded and a survey that also includes those who failed to respond. Non-respondent: person not contacted or who refuses to participate Self selection bias: extreme positions represented
  • Slide 17
  • 30/10/02 17 Response bias Errors that result from tendency to answer in a certain direction. Conscious or unconscious misrepresentation Types: 1. Deliberate falsification (why?)
  • Slide 18
  • 30/10/02 18 Why would people deliberately falsify data Appear to be what they are not Dont trust confidentiality Protect To end the interviewer quicker Average man effects
  • Slide 19
  • 30/10/02 19 Types of response bias continued: 1. Deliberate falsification 2. Unconscious misrepresentation
  • Slide 20
  • 30/10/02 20 Reasons for unconscious misrepresentation: Question format Question content Misunderstanding of question leading to biased answer Lack of time to consider answer fully Communication or semantic confusion other
  • Slide 21
  • 30/10/02 21 Types of response bias Acquiescence bias: individuals have a tendency to agree or disagree with all questions or to indicate a positive/negative connotation Extremity bias: results for response styles varying from person to person; some people tend to use extremes when responding to questions
  • Slide 22
  • 30/10/02 22 Types of response bias continued... Interviewer bias: Bias in the responses of the subject due to the influence of the interviewer Auspices bias: respondents being influenced by the organization conducting the study Social desirability bias: caused by respondents desire, either consciously or unconsciously to gain prestige or to appear in a different social role
  • Slide 23
  • 30/10/02 23 Reliability, Validity and Correlation are concepts which are easy to confuse because the numbers used to represent them are so similar This is because Validity and Reliability are largely based on the Correlation Statistic Validity and Reliability are closely related Correlation, Validity and Reliability
  • Slide 24
  • 30/10/02 24 What is Correlation? It is one way to measure the relationship between two variables It answers questions like: Is the relationship linear (straight-line)? Does the value of y depend upon the value of x or vice versa? How strong is the relationship, do the points form a perfect line? To measure the relationship we calculate the Correlation Coefficient Misconceptions: An insignificant result doesnt mean there is no relationship, it is just not linear. The Correlation Coefficient does not measure the slope of the relationship Correlation, Validity and Reliability
  • Slide 25
  • 30/10/02 25 The Correlation Coefficient The Correlation Coefficient has the following attributes: It can take a value in the range of -1 to +1 It is dimension less, i.e. its value is independent of the units of y and x Its value is independent of the measurement scales of x and y Methods to measure the correlation are Spearman (r) rho (nonparametric, ordinal data) Kendall Tau Correlation (nonparametric, ordinal data) Pearsons (Product Moment) Correlation (parametric, interval or ratio data) Examples of values of Correlation Coefficient (r): = +1 =0 -0.6 Correlation, Validity and Reliability
  • Slide 26
  • Slide 27
  • 30/10/02 27 What is Validity? Validity is concerned with whether we are measuring what we say we are measuring A measure is valid when the differences in observed scores reflect true differences on the characteristics one is attempting to measure and nothing else. X 0 =X T There are different kinds of validity Most of these use the correlation coefficient as a measure Correlation, Validity and Reliability
  • Slide 28
  • 30/10/02 28 What is Reliability? A Measure is reliable to the extent that independent but comparable measures of the same trait or construct of a given object agree. In research, the term reliability means "repeatability" or "consistency" Reliability is a necessary but not sufficient condition for validity A test is said to be reliable if it consistently yields the same results Example: Correlation, Validity and Reliability For instance, if the needle of the scale is five pounds away from zero, I always over-report my weight by five pounds. Is the measurement consistent? Yes, but it is consistently wrong! Is the measurement valid? No! (But if it under- reports my weight by five pounds, I will consider it a valid measurement)
  • Slide 29
  • Types of Validity? Correlation, Validity and Reliability Predictive Validity (Criterion Related) Test scores should corr. with real-world outcomes GMAT scores predict university success Convergent Validity Test should correlate with other similar measures GMAT should correlate with other academic ability tests Discriminant Validity Test should not corr. with irrelevant tests GMAT should not corr. with political attitudes Face Validity Items look like they are covering proper topics Math test should not have history items Construct Validity Construct validity can be measured by the correlation between the intended independent variable (construct) and the proxy independent variable (indicator, sign) that is actually used.
  • Slide 30
  • 30/10/02 30 Validity vs. Reliability? There are different conceptions of the relationship of Validity and Reliability which developed over time If a measure is valid it is also reliable? Illustrative Example: Target Metaphor Correlation, Validity and Reliability
  • Slide 31
  • 30/10/02 31 Types of Reliability? There are 4 types of Reliability:

Recommended

View more >