First training example

Three months ago I started a post-doc in physics education research at MIT in a group called RELATE, which stands for Research in Learning and Tutoring Effectively. [Before that I was a teacher, a furniture-maker/sculptor and a particle physics grad student, in reverse chronological order.] The work at RELATE, in the smallest possible nutshell, is focused on quantitative analysis in physics education research, especially for online learning environments. There is a lot of data mining here, informed by concept maps and cognitive models. Publications on the website tell the story better.

When I arrived at MIT, I inherited a project that had been worked on by at least two former postdocs (R. Warnakulasooriya and later C. Cardamone): applying Item Response Theory (IRT) as an assessment tool in our research. IRT is a psychometric method that allows you to measure parameters about assessment items (i.e. difficulty and discrimination of questions) simultaneously with the ability/skill of the person answering the questions. IRT extracts more information than classical test theory (CTT) and allows you to measure students on the same scale using different questions. It also helps you evaluate the quality of your items. There will be a lot more written about IRT in the future of this blog. One of the facets about IRT that is particularly relevant to mention is that it is not really designed to be used in a dynamic, noisy, online learning environment. IRT was developed for use in high-stakes testing, which is highly controlled. So we have a whole host of other interesting problems.

Because we are interested in the different ways that online learning takes place, I was paying attention when Stanford opened up three courses this Fall (Artificial Intelligence, Machine Learning, and Databases) to a wide on-line audience. Over 100,000 people enrolled in some of these. I started out curious about course format, but I quickly became even more interested in the content, especially Andrew Ng’s Machine Learning course. Much of this work is quite related to the IRT analysis I have been doing, and now my mind is spinning. Can we use machine learning algorithms to develop even better assessment tools than IRT, to…analyze student learning?

In order to keep track of some of these thoughts, I decided to start a blog (my first). The topic is the interplay of human learning and machine learning. On the simplest level, RELATE is interested in studying (and improving) human learning using machines, machines which help us analyze data that are obtained from the interaction of human students with other machines.  On a deeper level, Machine Learning can be used to learn about human learning from the same data.  This is less circular than it sounds. All three pairwise links between the nodes human, learning, and machine contain synergistic ideas.  In fact here is a fascinating workshop description from the NIPS 2008 conference titled Machine learning meets human learning.

That is the starting point.

Leave a Reply

Your email address will not be published. Required fields are marked *