Student information by the numbers

I alluded to the increase in information available about student learning from monitoring online interactions. I want to make this explicit and discuss the numbers a little bit. In a typical college course grade book, an instructor will record a dozen maybe fifteen homework grades, half a dozen quiz scores and, say, three exams. If each grade is on a scale of 0-100, we should allocate 7 bits for each grade (homework grades might not even be this detailed in practice). With 24 grades in the final gradebook, that makes 168 bits of information on the basis of which the student is finally evaluated.

In online interactions, students are almost always allowed multiple attempts, and sometimes they are given optional hints or can choose to do a practice problem first. The system can theoretically track every click, but practically speaking, we will at least keep track of the following information: number of attempts (3 bits), hints or intermediate resources used (3-6 bits), time-to-response (3-6 bits), problem-specific parameters quantifying the difficulty and discrimination parameters (10 bits). For each problem solved online, we thus record 20 bits of information. At 250 problems in a typical semester-long course, that makes 5000 bits of relevant information. Note this is more modest than the factor of 100 I have previously suggested, but I am being conservative in both the high and low estimates.

What is the difference between 168 bits and 5000 bits? I hope Daniel Kahneman won’t mind my repurposing an image he uses in his new book Thinking, Fast and Slow. Kahneman introduces the image to make a point about processes that occur very quickly in the brain, such as reading someone’s facial expression. Here is what happens if you have to compromise the eye’s extraordinary resolution for the limited information available in the two cases we are considering:

     

The image on the left contains about 168 pixels and the image on the right about 5000. I’ve cheated in that the pixel depth is actually 8 bits, i.e. grayscale not just black & white, so it would be more accurate to call them 168 byte and 5000 byte images. Interesting that at 168 pixels, you can tell that you’re looking at a face, whereas at 5000 you can tell that face is about to yell at you.

Leave a Reply

Your email address will not be published. Required fields are marked *