Article: Proof of Learning

Proof of Learning: Assessment in Serious Games

David Michael, Sande Chen [2005]

This article explains why it is not enough for a game to say that "it teaches" and leave it at that, but to include, like every other tool of education, a way of testing that the player actually learned something.

When a computer is used to test a student, what comes to mind are long sequences of multiple-choice questions (MCQs), which makes this the easiest choice for assessment in serious games. Sadly, MCQs are not always the best choice (I might go as far as to say that they are probably the worst choice), mostly because even if they are good to measure memorization and retention of facts, they fail at evaluating if the student is following a process correctly and also, during a game, this kind of assessment will be percieved as boring.

So, how can someone do a correct and useful assessment in the context of a serious game?

Currently, there are three main types of assessment that are used in serious games:

• Completion Assessment (The player completed the lesson?)
• In-Process Assessment (How did the player completed the lesson?)
• Teacher Evaluation (Does the teacher think the student understands the material?)

1) Completion Assessment

Translated to Serious Games, this could be if the student completed the game. The problem with this is that the player could have cheated or exploited holes in the system (which, as the article states, is something honored and rewarded between gamers, but not so between students). It's important to know whether the student learned the material or just learned the game and how to beat it.

2) In-Process Assessment

In Serious Games, this can be done by logging and tracking information while the student plays the game. For example, the time required to complete the lesson, the number of mistakes made, etc. Depending on the material that is intended to be learned, different kinds of information will need to be stored and different actions will need to be taken depending on them.

3) Teacher Evaluation

It's hard for a game to replace a teacher, so the game has to include tools to assist teachers in their evaluation, which can be homework control, grade tracking, reporting, etc. One example can be the inclusion of an 'Observer Mode' where the teacher and other students can actually watch the whole process.

As a game developer, implementing these methodologies can prove frustrating, because great care should be taken in maintaining the game fun. This is not an impossible task though, PIXELearning is a company that uses its own propietary engine to include pedagogy and assessment methodologies in their products.

One of these methodologies include a series of qualitative questions (I would assume at the end of a gaming session) such as: "You chose to do X. What was your basis for this decision? Why did you not choose Y?" So the teacher has more information available to judge how well the student really understands the material.

The article ends with a quote from Kevin Corti, former CEO at PIXELearning:

"[Serious games] will not grow as an industry unless the learning experience is definable, quantifiable and measurable. Assessment is the future of serious games".

Read the full article here

Michael, David; Chen, Sande. (2005). Proof of Learning: Assessment In Serious Games. Retrieved March 31, 2011 from Gamasutra.com: www.gamasutra.com/view/feature/2433/proof_of_learning_assessment_in_.php