This post is a short tangent from my usual theme of extending MMOs onto cell phones. My underlying desire with that theme is to improve the game experience for all players and to use these games to improve education (I'm designing educational games right now). I particularly like the multiplayer component of MMOs in the context of education, because so many of the things people want to learn to do involve interacting with other people and because anything we do has more meaning in a social context. So, I've been thinking about MMOs for education for awhile (particularly to teach languages). One property of using any kind of game for education is that assessment is built in -- the game knows if you're succeeding or not. However, I did not consider using these games to improve the sometimes-bane of every student's academic existence: standardized tests. These tests are, after all, basically the assessment without the learning.
Where is all of this coming from? My girlfriend is preparing to take the LSAT on Saturday, and has been studying for months. It's bringing back my own memories with the SAT and GRE. It has always struck me the degree to which a standardized test I consider arbitrary has influenced admissions committees' predictions of my future success or aptitude. I knew before college, before the SAT and GRE, that I wanted to design games "when I grew up." Now that I've been designing games for several years, and am much more aware of what the process involves, I find that my success and aptitude has very little to do with those standardized tests. Even my success in Computer Science, which was technically my major, had very little to do with these tests. A much better test would have been for me to actually design games (or, for computer science, write a simple program in simple a language with an unknown API/"toolbox").
I was reading some of Howard Gardner's work on multiple intelligences, and was struck by the following paragraph:
A shift away from standardized short answer "proxy" instruments to real-life demonstrations or virtual simulations. During a certain historical period, it may have been necessary to assess individuals by administering items that are themselves of little interest (e.g., repeating numbers backwards) but that are thought to correlate with skills or habits of importance. Nowadays, however, given the advent of computers and virtual technologies, it is possible to look directly at individuals' performances-to see how they can argue, debate, look at data, critique experiments, execute works of art, and so on. As much as possible, we should train students directly in these valued activities and we should assess how they carry out valued performances under realistic conditions. The need for ersatz instruments, whose relation to real world performance is often tenuous at best, should wane. (8)
For the most part, standardized testing has integrated computer technology by digitizing the same old tests. The big change in the case of the GRE is that the test is now adaptive on the computer -- it gives you harder questions worth more points if you correctly answer previous questions. However, now that we can construct virtual environments that simulate many of the tasks students are hoping to do someday, we can measure their aptitude at those tasks directly, instead of asking abstract, multiple-choice questions and then guessing at a presumably correlated aptitude.
It's not immediately obvious how a simulation would be constructed to test every field, but certain fields seem well suited: medicine, engineering, and aviation, for example. Even some softer skills like communication, leadership, resourcefulness, and creativity can be effectively simulated and scored in multiplayer virtual environments. Take communication, leadership, and ability to follow instructions -- all crucial factors in successfully organizing any team activity in MMOs. Admissions committees currently look to extracurricular activities, work experience, and personal statements for traces of these qualities in applicants, as they should. However, a test activity completed in a collaborative virtual environment could give even more insight and eliminate some of the guesswork. Players would need to repeat the test enough times to sufficiently reduce the variability introduced by the particular teammates or opponents the player had, so the test should probably be short.
The way the educational game space is proceeding right now, these virtual training/testing grounds will probably be applied first to teaching, perhaps in the context of a course, and only later to testing.
Another issue, of course, is that games are great learning environments for some students because some students enjoy games. But, the same has always been true of standardized tests -- they have always favored some students over others.
Gardner, H. (1998). A multiplicity of intelligences. Scientific American, 9, 19-23.
Another thought
(hi dan!) .. another interesting part of balancing "assessment" and gameplay is this: not all gameplay activities which are good for learning are also good for assessment of mastery.
Sometimes the gameplay gives you the chance to use what you've mastered, and that practice and use may be very helpful in adding depth or solidity to what has been learned, but there may not be a measureable game outcome that relates directly to mastery.
For instance: A strategic history simulation (say WWII), your knowledge of Russia's historical productive capacity, and of the nature of Russian winters will help you, but there are many other factors, so you chance success that simulation would be enhanced if you have that knowledge, but your success doesn't prove that you do.
Or... in a language-learning game, a game might provide a wonderful opportunity to use and practice socially some language that I've learned -- but "assessing" my conversational success doesn't directly relate to my mastery.
So... it may not make sense to assume (or demand) that all good learning games must also include asessment of "mastery" that can replace all other assessment.