Exam Grading
Response Analysis Guide
Response Analysis
This is used to see the distribution of the student’s responses. The correct response is indicated by an "*". For each item: line 1 gives the number giving that response; line 2 gives the percentage giving that response; and line 3 gives the point biserial coefficients. The coefficient is for the correct item (*) and the PBSR for other items that people chose if those had been the correct choices. For example, lets say that the correct item is B, 95.0% chose it with a PBSR of .254 meaning if you got this item correct you probably did well on the exam. However, for the same item, some people chose A which was incorrect. 2.5% chose it and it had a negative PBSR meaning that if A had been the correct item, those who chose it would have done poorly on the exam. It is reassuring to see negative PBSR for the items which are not the correct choices.
An example and definitions are below.
Item blank A(1) B(2) C(3) D(4) E(5)
1 1 0 0 1 1 111*
% 0.9 0.0 0.0 0.9 0.9 97.4
pbsr ***** ***** -.171 0.073 0.180
2 0 18 37 59*
% 0.0 15.8 32.5 51.8
pbsr -.300 0.111 0.115
Item: The question number.
Blank or Bad Responses: This reports who marked multiple answers for one question or left that question unanswered. It also helps us to see if there may be problems with the exam if there are a high number of bad responses. For example: it may be that a student only answered a couple of questions and left the rest blank or the answer key was filled out for 40 responses but the students only filled out 30.
Point Biserial Coefficients (PBSR):
It gives the PBSR for the correct item (*) and the PBSR for other items that people
chose if those had been the correct choices. For example, lets say that the correct
item is B, 95.0% chose it with a PBSR of .254 meaning if you got this item correct
you probably did well on the exam. However, for the same item, some people chose A
which was incorrect. 2.5% chose it and it had a negative PBSR meaning that if A had
been the correct item, those who chose it would have done poorly on the exam. It is
reassuring to see negative PBSR for the items which are not the correct choices.
Item Analysis
This is used to see how well the exam was written. You first look to see how many students got an item correct and then you look at the PBSR (Point Biserial Coefficient). This will tell you the relationship (correlation) between a specific item and the overall test score. A positive PBSR means that students who answered that item correctly had overall higher scores on the exam. A negative PBSR means that students who answered the item correctly had lower scores on the exam. (The opposite, of course, is also true: For positive PBSR, students who answered the item incorrectly did worse on the exam. For negative PBSR, students who answered incorrectly had higher scores on the exam).
If you left an item unanswered, double marked or the scanner could not read the marking, it will state Item Deleted By Instructor. The exam will then be scored by one fewer point. Meaning if the exam was worth 30 possible points, this exam will only be worth 29.
An example and definitions are below.
Item Key WT PBSR # Correct % Correct
1 E 1 0.180 111 97.4
2 C 1 0.115 59 51.8
3 B 1 0.180 111 97.4
4 Item deleted by instructor
Item: The question number.
Key: The answer given on the answer key.
WT (Weight): How much each question is worth.
Point Biserial Coefficients (PBSR):
This tells you the relationship (correlation) between a specific item and the overall
test score. A positive PBSR means that students who answered that item correctly had
overall higher scores on the exam. A negative PBSR mans that students who answered
the item correctly had lower scores on the exam. (The opposite, of course, is also
true: For positive PBSR, students who answered the item incorrectly did worse on the
exam. For negative PBSR, students who answered incorrectly had higher scores on the
exam).
# Correct: How many students got this question correct.
% Correct: The percentage of correct answers.