tools of research
TRANSCRIPT
TOOLS/INSTRUMENTS OF
RESEARCH
1. CAHYA ADY P. (10420184)2. DWI PRASETYO A. (10420192)3. TAUFIK SETIAWAN (10420209)4. ANDRIE PRASETYO (10420225)5. TRI AGUS SETIAWAN (10420380)
1. QUESTIONNAIRE2. OPINIONNAIRE/ATTITUDE SCALE3. TEST
Kind of tools of research
A technique for obtaining information from subjects.
A series of question ask to individuals to obtain statistically useful information about a given topic.
Questionnaire?
relativelly economical
has the same question for all subject
can ensure anonimity
Why must be Questionnaire?
How to make Questionnaire
Requirements
Determine the type of information
Limit responses
Figure out who, what, when and where you're going to distribute your questionnaire.
Straigh forward& clear
Quantity
State introduction& purpose
Simple, short & in a page
Double check
Profesional, easy read, & understandable
A. Decision about question contentB. Decision about question wordingC. Decision about form of response to the
questionD. Decision about the place of the question in
the sequence
Guide for constructing questionnaire
types of questionnaire
Types Types
Closed-ended
Open- ended
Contingency
Matrix
No option answer/predefined categories are suggested.
A question that is answered only if respondent gives a particular response to a previous question.
identical response are assignned to multipe questions.
Respondents’ answers are limited to a fixed set of responses.
Provide for marking a yes or no, a short response, or
checking an item from a list of suggested responses. Why did you choose your graduate work at this
University?
Rank:
a. Advice of friend.
b. Scholarship aid.
c. Reputation of University.
d. other
CLOSED-ENDED
Completely unstructured, free response in the respondent’s own word, no clues are given.
Why did you choose your graduate work at this University?Answer :_________________________________________________________________________________________________________
OPEN-ENDED
Differences
Closed-Endeda. highly structuredb. Generate frequencies of responsec. Amenable to statistical treatment &analysisd. More focuse. To the point
Open- endeda. Unstructuredb. Suit for complex issuec. respondents can explore more their thoughtsd. Not limited of pre-set categoriese. Lead to irrelevant & redudant information
chracteristics
•Who are interviewed and their background /experiences.
Factual (Demographi
c)
•What they do, or did in the past.
Behavioural
•Attitudes, opinions, beliefs, interests, and values.
Attitudinal
Types Gained Information of Questionnaire (dornyei, 2003, PP.8-9)
1. Significant topic2. Single source question3. Short, simple and understandable4. Attractive in appearance, neat arrangement
and clearly duplicated5. clear and complete direction, simple and clear
questions, and single idea or concept 6. good psychological order and it proceeds
general to more specific responses7. objective questions8. easy to tabulate and interpret
Characteristics of a Good Questionnaire
The question of content validity is: “Do the items sample a significant aspect of the purpose of the investigation?”
Clearly defined meaning in
questionnaires
Get help from colleagues or experts from the field
of questions
The panel of experts can rate the instrument to know how
effectively it samples significant aspects of its purpose
Estimating the predictive validity of a questionnaire by a follow-up
observation of respondent behavior at the present time or
at some time in the future
Validity and reliability of questionnaires
The information form that attempts to measure the attitude or belief of an individual.
How people feel, or what they believe is their attitude. But it’s difficult, if not impossible, to describe and measure attitude. Researchers must depend upon what people say are their beliefs and feelings. This is the area of opinion. Through the use of questions, or by getting people’s expressed reaction to statement, a sample of their opinions is obtained.
OPINIONNAIRE/ATTITUDE SCALE
1. Thurstone Technique
( equal – appearing Interval Scales)
2. Likert Technique
( Summated Rating
Scale )
Types of Attitude Scale
The list of statement is given to the subjects, who are asked to check the statements with which they agree. The median value of the statements that they check establishes their score, or quantifies their opinion
1. Thurstone Technique
The second method, the Likert Method of Summated Ratings, which can be carried out without the panel of judges, has yielded scores very similar to those obtained by the Thrustone method. The coefisient correlation between the two scales was reported as +.92 in one study. Since the Likert Scale takes less time to construct, it offers an interesting possibility for the student of opinion research.
The Likert scaling technique assigns a scale value to each of the five responses. Thus, the instrument yields a total score for each respondent and a discussion af each individual item, while possible, is not necessary.
2.Likert Technique ( Summated Rating Scales)
a. Strongly agree 5
b. Agree 4
c. Undecided3
d. Disagree2
e. Strongly disagree1
Starting with a particular point of view, all statements favoring this position would be scored:
Scale value
For statements opposing this point of view, the items are scoresIn the opposite order:
Scale valuea. Strongly agree 1b. Agree 2c. Undecided 3d. Disagree 4e. Strongly disagree 5
1. Heaven doesn’t exist as an actual place or location
2. God sometimes sets aside natural law, performing miracles
3. Hell does not exist4. The devil exists as an actual
person5. God is a cosmic force, rather
than an actual person6. There is a final day judgment for
all who have lived on the earth.
ilustrationa b c d e
a b c d e
a b c d e
a b c d e
a b c d e
a b c d e
If the opinionnaire consisted of 30 statements or items, the following score value would be revealing:
30 x 5 = 150 Most favorable response possible
30 x 3 = 90 A neutral attitude30 x 1 = 30 Most unfavorable attitude
The scores for any individual would fall between 30 and 150; above 90, if opinions tended to be favorable,below 90, if opinions tended to be unfavorable to the the given point of view.
Definition :
a method of measuring a person’s ability, knowledge, or performance in a given domain.
TEST
1. ACHIEVEMENT TEST:attempt to measure what an individual has learned – his or her present level of performance. Most test used in schools are achievement tests. Achievement test scores are used in placing, advancing, or retaining students at particular grade level. Achievement tests scores used to evaluate the influence of course study, teachers, teaching methods, etc/
TYPE OF TESTS
2. The Proficiency TestThe proficiency test also measures what students have learned, but the aim of the proficiency test is to determine whether this language ability corresponds to specific language requirements.
There are some samples of proficiency tests: 1. Placements test
The student proficient enough to enter intermediate course or it is better to place him or her in basic course.
2. The reading- knowledge tests for doctoral candidates
Example: Is the student able to read professional literature in another language with a specific level of accuracy?
The progress test measures how much the student has learned a specific course of instruction. The tests that the classroom teacher prepares for administration at the end of a unit or the end of a semester are progress tests.
The Progress Test
Multiple choiceArranging sentenceFill in the blankMatchingTrue or falseShort answer
Types of test items
Constructing multiple- choice items
1. Design each item to measure a specific objectives
2. State both stem and option as simply and directly as possible
3. Make certain that intended answer is clearly the only correct one
4. Use item indices to accept, discard, or revise items.
Analysis ofMultiple-Choice Tests
1. To develop the quality of test items through the students’ answers for each item.
2. To find out whether the test items will be accepted, discarded, or revised.
1.IF (item facility or item difficulty)2.ID (item discrimination and sometimes called
item differentiation) 3.Distractor
How to analyze?
IF (item facility)
The extent to which an item is easy or difficult for the proposed group of test-takers.
Brown (2000) defines item facility as the proportion of students who answered a particular item correctly.
The proportion is shown in index of IF.
The formula used to get the item facility
The number of test-takers answering the item correctly
IF ----------------------------------------------------------------------------- The number of test-takers
e.g. if you have an item on which 13 out of 20 test-takers respond correctly, your IF index is:13 divided by 20 equals .65 (65%)
The Classification of IF Item
0.00—0.30 The item is difficult0.31—0.70 The item is medium0.71—1.00 The item is easy
The extent to which an item differentiates between high and low ability test takers. An item on which high ability students and low ability students score equally well would have poor ID because it didn’t discriminate between the two groups.
ITEM DISCRIMINATION (ID)
The formula for calculating ID is
High group # correct – low group # correct ---------------------------------------------------
½ x total of your two comparison groups
Item #23 # Correct # Incorrect
High-ability Ss (to 10) 7 3
Low-ability Ss (bottom 10) 2 8
Using the ID formula (7-2 = 5 ÷ 10= .50), you would find that this item has an ID of .50, or a moderate level.
The Classification of ID Item
0.40—1.00 The item is accepted0.30—0.39The item is accepted but it needs
revising0.20—0.29The item must be revised0.19—0.00The item is discarded
(Safari; 2005: 27)
The functions of ID are:
To increase the quality of an item of test through the empirical data. Based on the index of the ID, each item will be found out to be good, be revised, or be discarded.
To find out how far each item can detect and distinguish the students’ ability, that is students who understand or do not understand the material taught by the teachers.
Is one more important measure of multiple choice item’s value in a test, and one that is related to Item Discrimination (ID)
The pattern is obtained by calculating the number of test-takers who choose the option of test item or who do not choose any options. The distribution pattern will show whether or not the distractor works well.
DISTRACTOR
Example:
Choices A B C D E
High-ability Ss (10) 0 1 7 0 2
Low-ability Ss (10) 3 5 2 0 0
Note: C is the correct response
The characteristics of distractor is:
the distractors should be brief and as homogeneous as possible,
the distractors should be plausible, use at least 3 distractors to reduce the chance of
guessing the correct answer,avoid distractors that provide clues, the distractors should embody misconceptions,
partly correct answers and common errors of fact or reasoning.
TEST LAYOUTClear Instruction
What to do Appropriate response
Time allowance How many items to attempt
Example:
Test consists of 20 questions. Read each question carefully. In the button beside each answer, check the
letter that indicates the correct answer for each question. 20 minutes are given to complete the test.
TEST TIMINGIt will determine how ready, motivated, and
capable a student might be.The best time is between 11 a.m. to 1
p.m.Time allowance should be told to the
students.One should consider the level of difficulty in
setting out the time allowance.
SCORINGFigure out the objectives and determining the
weight of each item.
Percent of
Total Grade
Possible Total
Correct
Listening 30% 20 items @ 1.5 points each = 30
Grammar 40% 20 items @ 2 points each = 40
Reading 30% 20 items @ 1.5 points each = 30
Total 100