Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Even that's not punishing enough, since it becomes straightforwardly strategic to guess if you expect your "average" score to fail. You really want wild guesses to be negative expectation, AMC-style. That doesn't exactly solve the problem, but at least becomes less exploitable—if you're competent to optimize the number of guesses you make dynamically during a test, you probably less likely to need to.


I think it's okay for wild guesses to be zero expectation; if you look at it from an information-theory perspective, leaving the question blank and wildly guessing both communicate the same thing, "I don't know." What's the benefit of incentivizing test takers to leave questions blank instead of wildly guessing? I get that it feels better to discourage guessing, but how does it improve the accuracy of the test as a measure of understanding?

And if you can rule out at least one answer, that communicates that you know something about the material under test - provided, of course, the test has no joke or otherwise implausible answers, which appears to be true of the Regents test that I posted above. Take question 6, for example, which asks whether the days of the month are best described as "(1) integers (2) whole numbers (3) rational numbers (4) irrational numbers". Suppose you forgot the distinction between integers and whole numbers, but you knew that the other two answers were incorrect, and so you randomly picked one of the first two options. Doesn't that convey understanding of part of the material under test, such that a score of a positive fractional point in expectation is reasonable?

(A more logistically complicated system would be to permit students to mark any answers they think might be correct and therefore remove the "in expectation" part. If you don't select the correct answer, you get no points, but if you select n answers including the correct answer, you get 1/n points, and leaving a question blank is considered equal to selecting all the answers. But the cognitive overhead of this system probably makes it not worthwhile.)




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: