Providing Students with Positive Failure Feedback

I have written in the past about the importance of making failure okay, and indeed par for the course, in education. The fact that games make failure normal, acceptable, and even fun is one of the many aspects of play that has drawn me to game-based learning, gamification, and gameful teaching/learning. But even when I tell students that it is okay to fail and build in a do-over system into my classes, it is still a struggle to get students to buy into the idea that failure is an acceptable and necessary component of learning. I am currently re-reading Jane McGonigal’s Reality Is Broken and have spent much time studying and contemplating her chapter on failure. McGonigal points out that there is a biological imperative to our avoidance of failure:

It’s to our evolutionary advantage not to waste time and energy on goals we can’t realistically achieve. And so when we have no clear way to make productive progress, our neurological systems default to a state of low energy and motivation. (70)

This would certainly explain why so many students choose avoidance over failure. But even when I believe that I have provided my students with clearly attainable goals and lavished them with multiple streams of formative feedback, sometimes even the most capable student will give up and become disengaged. It could be a problem with my perception of the assignments I am creating, the feedback I am giving, and the mechanisms for self-direction I have built into those assignments; perhaps my instructions are not as clear or the goals as obtainable as I believe them to be from the students’ perspectives. Perhaps it’s a problem with students’ self-efficacy beliefs or their ability to persist in the face of academic failure (which is certainly more life-threatening than virtual failures in a game). I can certainly try to address the former, but I am not sure what more I can do to remedy the latter. Another component that is within my control is the type of feedback that I provide students when they fail.

One of the reality fixes that McGonigal sees games providing is that of “fun failure:” “The right kind of failure feedback is a reward. It makes us more engaged and more optimistic about our odds of success” (67). She provides several examples of this in games and explains how studies have shown  that players exhibit the most heightened positive emotions, such as excitement, joy, and interest, immediately after they have experienced failure. Why is this? McGonigal believes it is because games allow us to fail spectacularly and actively. In a game, failure is not something that just happens to you; it is not beyond your control. If you fail, it is because you did something wrong and you know it. And when you do fail, it is communicated to you immediately and usually in a way that is so celebratory (via sounds and visuals) that it renews feelings of positive engagement. According to McGonigal, the trick to accomplishing this magical failure reaction is pretty straightforward:

[Y]ou have to show players their own power in the game world, and if possible elicit a smile or a laugh. As long as our failure is interesting, we will keep trying–and remain hopeful that we will succeed. (67)

I am still grappling with how to achieve the first aspect of this method in my formative and summative feedback. But, in the meantime, I’ve decided to try to at least achieve the second aspect by attempting to elicit a smile or a laugh from my students when they fail.

I decided to start playing with integrating some positive failure feedback into my current short-term online FYC class. The easiest thing to start with, I decided, would be the quizzes, since Blackboard offers a way to provide immediate feedback to a student based on whether they answer a question/problem correctly or incorrectly. I had already added some positive feedback when students answered some of the most difficult questions correctly. This is something that seemed natural to me at the time that I was creating the quizzes. But I am not sure why it has never occurred to me to also provide positive feedback when students answer those questions incorrectly. For some reason, this seems counterintuitive to my teacher senses. But, if you think about it, it actually makes much more sense than providing feedback when a student gets a question/problem correct. If a student gets a question/problem correct, they don’t really need us to give them kudos: success should be reward enough. It is when a student has failed that they need the most encouragement. So, I decided to do this by trying to do two things: make light of their failure (it’s really not the end of the world that you got this question/problem wrong) and, by extension, make them smile or laugh (so, just dust yourself off and try again and, if you happen to fail again, you’ll get a good laugh out of it). I chose to do this by selecting .gifs featuring the minions from the Despicable Me franchise either failing miserably (and spectacularly) or comforting each other after such a failure. They are, after all, immediately recognizable, have a reputation for screwing up, and make us feel all warm and fuzzy because of their persistence and unfailing hope and happiness. I embedded the .gifs in the feedback box for incorrect answers on what I considered the most difficult questions in each test pool, choosing a random approach, since randomness is another way in which games reward the brain. My hope is that, should a student begin to become unmotivated in the face of failing to answer a difficult question correctly, a funny .gif will both make them smile and encourage them to try again with more confidence in their ability to succeed or, at the very least, get a good laugh if they fail again.

For future classes, I would like to try other ways of integrating positive failure feedback, especially with writing assignments. Even though students can attempt a writing assignment multiple times, for students who lack basic writing skills, it can often take anywhere from three to six attempts to get a piece of writing to an acceptable level and this can become extremely frustrating for them. If I can determine a way to make them feel more in control of their success and more empowered by their failures, I can perhaps keep them motivated and engaged.

I would love to hear readers’ thoughts and ideas on ways to provide positive failure feedback to students.

Advertisements

Turning Your Class into a Game, Part 3: Rewarding Effort

image2

In my last two posts, I covered two aspects of turning your class into a game: creating the experience and designing experience systems. In this post, I’m going to cover the third aspect: rewarding effort. In games, all effort is rewarded and failure is not punished. In fact, failure is built into games. No one ever plays Mario Kart or Assassin’s Creed without failing–multiple times. This, as has been pointed out by several GBL advocates, is one amongst several reasons why games get learning right and schooling gets learning wrong (or, at least, has poorly designed it). In addition to de-stigmatizing failure, games reward every effort on the part of the player. Every effort. No matter how small. No matter if the effort leads to ultimate success or abysmal failure. Not only do games reward all effort, but, as James Paul Gee points out in What Video Games Have to Teach Us about Learning and Literacy, they reward effort based on the amplification of input principle. In this design principle, a little input results in a lot of output. By rewarding the player with mega-feedback and mega-output, the game encourages them to put forth even more effort in the hopes of receiving ever larger and larger returns. Again, this stands in stark contrast to how schooling responds to effort. So, how can we apply these principles to the classroom? That’s the question I’ll try to answer in this post, though this is an aspect of games-based learning that I have only recently begun experimenting with myself, so what I’ll offer are some basic principles, as gleaned from well-designed games, and a few ideas based on things I have done or plan to do in my classes.

One way that games reward effort is, of course, with points (XP). This is usually the first thing that people think of when they think of GBL and gamification: giving players/students points for doing things. But points are only one way in which games reward player effort. They also reward effort via achievements. Achievements can be almost anything that has value within the game: tools, clothing/armor, virtual money, powers, bonus content, advantages over other players/NPC’s, etc. In general, there are two types of achievements in games: measurement achievements and completion achievements. Completion achievements are earned simply for completing a task, while measurement achievements are awarded based on the degree and/or proficiency to which the task is completed and are evaluative in nature. A good example is the star rating system in Angry Birds, in which the number of stars you receive for destroying the pigs’ structure in each level depends on how well you did so (evaluating aim, accuracy, speed, and number of projectiles you used). In The Gamification of Learning and Instruction, Karl Kapp recommends awarding completion achievements for boring tasks and measurement achievements for challenging and interesting ones.

IMG_0441

Achievements can be expected or unexpected. Expected achievements, according to Kapp, encourage goal-setting and self-evaluation, as players seek to earn achievements that they know are available. Unexpected achievements, on the other hand, encourage exploration and creative gameplay; as a player discovers an unexpected achievement, they become curious about other achievements that might be hidden in the game and actively seek them out. A good example of unexpected achievements are Easter eggs, which I discussed in a previous post. Kapp recommends using unexpected achievements sparingly, but I’m not sure that I agree with him. I think that using both expected and unexpected achievements as much as possible will allow you to target and encourage both types of behavior, and unexpected achievements will offer more challenges for those students who crave them.

There are four general types of achievements in games: status (badges, character classes, etc.), access (to places and items that other players don’t have access to), powers (extra abilities and advantages), and loot. There are several types of loot, including money, goods, bonuses, and time. Money can be used to purchase apparel and tools/weapons for your character or items within the game world. Awarding players money encourages autonomy, creativity, and problem-solving, as they must consider what items to purchase based on both their current and future needs. For example, in the game that I designed for my argumentation and debate class this past Spring based on classical Greek institutio rhetorica (schools of rhetoric), teams of students earned (virtual) gold for participating in in-class activities; they could then use this gold to purchase “favors” from their patroness (me) such as the ability to select which side they debated in an upcoming debate or what order their debate would be held in. In the “Murderers and Mad(wo)men” game that my English 102 students played, they earned money for completing writing assignments and for helping out their guild members with their “cases”; the players could then use this money to purchase virtual investigative tools for their character. The number and cost of the tools their character owned determined their character’s status within the world of the game in terms of renown within their field.

Goods are a second type of loot. The investigative tools that my students purchased in “Murderers and Mad(wo)men” is one example. Goods can be used to personalize and/or strengthen a character or allow the player to play “smarter.” In my upcoming FYC game, players will earn potions for various efforts (peer review, attendance, commenting on peers’ blog posts, etc.); there are three different colored potions, each earned for a different type of effort, some completion-based and some measurement-based. The potions can be combined to attain various kinds of powers, which give the players advantages within the game (extended deadlines, bonus XP, skipping tasks, etc.), with each power requiring a unique combination of potions–the more advantageous the power, the more complex the potion combination required. And once a potion has been used to attain a power, it is used up, so, with some potions scarcer than others, the students will have to think carefully about which powers are most needed at the moment and which might be needed later on. So, like money, goods encourage creativity and problem solving, as well as goal-setting.

Bonuses are also effective ways to reward effort, whether in the form of bonus points or items, because they often allow players to catch up with other players or recover from an especially debilitating failure. A good example of this is found in Mario Kart, where the best weapons are often dropped at times when and in places where the players at the back of the race can pick them up. Students who start a class late (either literally due to late registration or figuratively because they chose to ignore early assignments due to lack of interest or competing commitments) or who get behind later in the term may become demotivated if they feel that it’s impossible for them to catch up with everyone else or make enough progress to pass to class. Having bonuses that allow these students to get back on track may help keep some of them from giving up. I’m attempting to address these students in two ways in my upcoming FYC class game. For one, a couple of the powers that can be attained by combining potions include earning double and triple XP on quests and the ability to skip certain tasks. I have also designed a couple of bonus quests that students can complete, adding the bonus XP earned for doing so to their current quest XP in order to help them level up to the next quest (one of the rules of the game is that players have to earn at least 50% of the total possible XP for a quest in order to move on to the next quest). Like bonuses, time can be used as a way to gain advantages over other players or over the game, again allowing players who get behind a chance to redeem themselves by either slowing down or speeding up the game for themselves or for others.

Whatever types of achievements you design to enhance the experience of your players/learners, have what game designers term a trophy room–a place where students can (re)view and relive their glory, whether virtually or physically. And try to tie achievements to activities that are rewarding in and of themselves. Too often, teachers believe the lie that we have to purchase student effort by assigning (subtractive) points to everything. This practice creates a self-fulfilling prophecy wherein students begin to buy into the idea that only things with points attached to them are worth doing. And that is ultimately the message we send when we attach rewards to some things and not to others (these things are worth doing, these are not), even though we scream about how lazy students are because they won’t read the assigned textbook or essay (but we haven’t designed any explicit reward system for doing the reading, while everything else is replete with punitive, external motivators). Within well designed games, rewarding effort is not about attaching extrinsic carrots to everything. An excellent example of this is the “student in peril” component of the Lego Harry Potter franchise. In each level, there is a student hidden somewhere; if you manage to find that student, the game rewards you by playing special music, having the student dance around while other students cheer, and providing you with a celebratory announcement. That’s it. No points. No extra powers. If you happen to rescue all of the students in peril (there are 50), then you earn the status of having rescued all of the students in peril. There’s no real extrinsic value in doing so. Yet, I have dedicated more hours than I really wish to think about trying to locate and rescue all of the students in peril, even replaying levels I’ve completed in order to do so. Why? Because it is a challenge (the students aren’t easy to locate, so it takes effort and skill to do so) and it feels good knowing that I have the potential to overcome this challenge the more I engage in the process (once you’ve located one student in peril, the chances are good you’ll be able to locate another and then another). And the fact that I’m not getting anything out of it actually makes it even more motivating, strangely enough (although Daniel Pink has proven this is actually not that strange).

 

An achievement is just that–a) a thing done successfully, typically by effort, courage, or skill; and b) the process or fact of doing that something successfully. In order to be most effective and to encourage intrinsic motivation, achievements need to be part and parcel of an experience in which the effort, courage, and skill required to do something successfully and the process and fact of doing it successfully are the rewards most valued, both by the teacher and the students. If you think about it, I am, in fact, receiving something for rescuing the students in peril in Lego Harry Potter: positive acknowledgment and feedback from the game (reward for effort) as I engage in the process of locating them and the pride and self-confidence that comes with doing so successfully. Points are not the only, or even the most powerful, form of rewards available to teachers. Positive feedback, acknowledgement, pride, and self-confidence are all types of achievements that belong in any classroom, whether it’s been designed as a game or not.

What do you think? How can achievements be used most effectively in the classroom? What kinds of achievements work best in the classroom? How can we best balance extrinsic and intrinsic rewards? These are questions that I am considering as I begin the process of integrating achievements into my classes and, I believe, some of the most important ones to consider as we turn our classrooms into game spaces.

Turning Your Class into a Game, Part 2: Experience Systems

Image
Warning: Designing effective experience systems is not easy!

In my last post, I began a discussion of how to turn your class into a game by addressing the most important component of a game: the experience. In this post, I’m going to focus on designing and integrating effective experience systems into that experience. Experience systems are what many people think of when they hear the term gamification: experience points, leaderboards, etc. But truly effective experience systems are so much more. I’ve discussed these systems and my struggles to integrate them effectively before (see “Experience Points in the Classroom: Back to the Drawing Board”), but I think that it’s important that the discussion about how best to use experience systems in the classroom continue. These systems are often viewed as not only a ready-made and easy-to-install substitute for traditional grading systems, but a way to automatically increase student engagement and motivation, neither of which is true. So what are experience systems, if not a game-based version of grades, and, if they increase gamers’  engagement and motivation so effectively, why are they so hard to integrate into the classroom?

Firstly, experience systems are methods for tracking both player progress through the game and their accomplishments and skill level. It’s important to remember that players don’t play the game to earn points, loot, or skills; rather, points, loot, and skills are byproducts of engaging in the experience in which the game immerses the player. This is why you should start the game design process by first designing the experience, then considering your experience systems. There are two major types of experience systems: experience points (XP) and levels. XP are a form of extrinsic reward system that are most often utilized in gamification and many teachers who are attempting to gamify their classes make the mistake of simply replacing grades with XP or adding an XP system to the class that is indirectly tied to grades (I’ve been guilty of both). This is a mistake because game designers don’t rely on XP alone to drive player motivation; instead, they purposefully and thoughtfully utilize both XP and levels in ways that aim to trigger both extrinsic and intrinsic motivation. Another mistake teachers may make is using XP as a substitute for traditional grades, but doing so in a way that simply turns XP into grading by a different name. The key characteristic of XP is that they are additive rather than subtractive. In other words, whereas traditional grading systems punish failure or a lack of effort by subtracting points from a predetermined grade (such as 100 points on an exam), players start a game with zero points and earn XP for every effort (whether it is entirely successful or not). Using XP in the classroom requires a reconceptualization of assessment that recognizes all effort and does not punish failure, one that reflects the degree of correctness and not absolutes. You should also balance the difficulty of the task with the amount of XP that the students can earn for attempting it. In the language of traditional grading, instead of all exams being worth 100 points, the number of points possible on each exam should become exponentially larger as the difficulty of the exams and the amount of knowledge, understanding, skills, and effort required to complete them increases. And when it comes to assigning XP to tasks, think BIG! Instead of 100 points, make something worth 1,000 points! The value of each point is still only relative to the points system itself, but you would be surprised by the difference in response from students when you raise the number values within that system. Lastly, for XP to be truly effective, you should update them immediately and frequently; one of the ways in which games reward the brain is via clear, immediate, and frequent feedback. The more rapid the feedback (whether it is XP, achievements, or narrative), the more likely the student is to link the feedback to an action/behavior on their part, synthesize and analyze the causal relationship between the two, and make adjustments accordingly.

The next type of experience system is levels and there are three types of levels: experience, progress, and difficulty. Again, all three of these types of levels should occur simultaneously. Experience levels are tied directly to XP and establish both short-term goals (leveling-up to the next experience level) and long-term goals (reaching the highest experience level). Players usually earn both status and external rewards of some kind (bonus XP, loot, etc.) when they reach a new level. When integrating experience levels, have a way for students to visualize their current level and how many XP they need to earn in order to level-up. Again, this is a type of feedback, so it needs to be updated rapidly and frequently. Progress (sometimes called game) levels help structure the game narratively and establish short-term goals for players, as each level presents new information and a new set of skills to master before moving on to the next level. This is just the game version of scaffolding or organizing learning into units, something most teachers are probably already doing. The difference with game levels is that the player’s progress is visible at all times (via a progress bar), allowing them to easily measure and visualize how far they’ve come since they started playing and how far they have left to go in order to win. Again, progress through the levels (or units) is a form of feedback that helps students develop metacognition (self-monitoring of effort and learning progress) and should be kept up-to-date.

The final type of level is difficulty levels. There are generally three difficulty levels: easy, intermediate, and hard or advanced. Having multiple difficulty levels allows you to differentiate learning; increases student autonomy; improves the chances of achieving the zone of proximal development (because each student can select the task that best correlates with their current skill level); and increases the re-playability of the game (if students can re-play a level at different difficulty levels). I’ve heard teachers say that, given a choice, students will opt for the easiest task, but, in my experience, that is not true. Though some students may do so (those trying to avoid failure, for example), most students are very accurate judges of their current ability level and relish a good challenge that will push them beyond that level if the classroom environment supports risk-taking and and if they are engaged in an experience that is relevant and interesting to them (both of which are present in effectively-designed games). It may take extra effort on the front end to develop different tasks of varying degrees of difficulty for each game level and then keeping track of which level each student is working at, but it will be well worth it if it means that more students are working in the zone of proximal development. And not all game levels have to be diversified; you could taper off explicit difficulty levels so that, eventually, all students are working on the same task, but one that is designed in a way so that each student can work at their own skill level (such as a problem-based project). If a three-level system is too complicated to schedule or difficult to design, then you could just use a simple two-level system: normal gameplay for the majority of students and advanced gameplay for those who want/need a more challenging task/quest (like those extra credit projects that students are always begging us to let them do). Either way, having different difficulty levels for students to choose from will increase the motivational and engagement factors of your game and is an important component of a well-integrated, multi-tiered experience system (experience points, experience levels, progress levels, and difficulty levels).

Because the most effective game strategically integrates all of these components, designing your experience systems is a difficult and potentially time-consuming task, especially when you factor in creating spreadsheets that allow you to update the experience systems so that students can see and monitor their progress. I have found it to be a trial-and-error process, with failure almost certain (either in terms of quality of design or in terms of how effectively your systems impact student motivation). It’s important to keep an open dialogue with students about the effectiveness and impact of your experience systems so that you can make any necessary changes quickly, before they have the potential to become a de-motivating factor that cancels out all of the hard work you’ve done in designing an engaging learning experience. But implementing experience systems into your game is essential to providing students with rapid, frequent feedback; to encouraging them to set and achieve short- and long-term goals; and to inspiring competition, whether it’s player vs. player, player vs. self, or player vs. the game. Students must have a way to visualize their current status in the game environment, how much progress that represents, and how close they are to winning.

I certainly don’t have all of the answers when it comes to designing effective experience systems, but I am more than willing to share my own ideas and experiences, if you have questions.

Next in the series, I’ll address different methods for rewarding effort in games.

 

Teaching Revision vs. Editing

image courtesy of Alex Pang http://flic.kr/p/8AJ566
image courtesy of Alex Pang http://flic.kr/p/8AJ566

My most recent post dealt with postmortems on student writing. In a related line of thinking, I have been considering more effective methods for teaching students the differences between revising and proofreading/editing a piece of writing. I often hear composition teachers express frustration with students because they insist on conflating editing with revision, despite the teacher’s best efforts to teach students the difference. It’s an issue that I have also struggled with and it is probably the one aspect of writing instruction to which I have yet to find a satisfactory solution. This term, I am teaching the first-semester course of my department’s First-Year Composition class. Since it is Spring, and this course is generally taken by students in the Fall term, the students in my class are taking the course out-of-sequence, meaning they have either failed the class once (or more times) before or were required to take our remedial writing class in the Fall. Both scenarios indicate that these students are, generally speaking, weaker writers than those who take the course in sequence in the Fall. Since weak compositions are, in my experience, more a reflection of lack of effort and revision/editing skills than lack of ability or writing proficiency, I decided to focus on teaching the students better revision/editing skills.

I decided to try to get to know more about the students in the class by having them write a literacy narrative as their first piece. I hoped that the literacy narrative would provide two things: some insights into the students’ experiences with and feelings about writing and a platform for explicitly teaching the writing process by requiring multiple drafts that focused on different writing processes.

The overwhelming majority of students chose to focus their literacy narrative on negative experiences with writing, either at the secondary level or at the college level. These experiences, while painful for the students to write about and, sometimes, for me to read about, were, I believe, cathartic for the students and extremely helpful in showing me, from the students’ perspectives, what methods do and do not work. One common factor among these negative experiences were feelings of inadequacy as a result of being singled out or overly criticized by their writing teachers. One student told a story of being unable to even begin writing an impromptu essay in their high school English class and feeling overwhelmed by being the only student in the class who was struggling to get started. They were then called on by the teacher to share their essay with the class and decided to improvise, despite not having a single word written down. When the teacher called the student out for not having written anything and speaking extemporaneously, the student broke down in tears and experienced what she termed “permanent writer’s block.” Another student wrote of their first college writing class, describing a grueling essay assembly line of in-class writing with no opportunities for revision after the pieces were graded. They elaborated on one incident in which the teacher marked off because the student had used “you” in their essay; in an effort to not make the same mistake, the student spent extra time on the next essay, making sure not to use “you,” only to receive deductions again for using “you.” When the student approached the instructor and pointed out that they had not used “you,” the teacher responded, “You implied it.” With experiences like these, it is easy to see why so many of our students see their composition classes as either a nightmarish torture chamber or a game filled with arbitrary rules, which they have no hopes of winning.

The thing is, most students did not write this openly or use these kinds of illustrative examples the first time they wrote their literacy narrative. Typically, most students submitted bare bones pieces, some no longer than a paragraph, full of vague and abstract generalities. Normally, I would spend the majority of my feedback addressing this lack of content and the need for examples and supporting details and use the minimal marking method to mark but not correct errors in grammar and mechanics. It would be up to the student to address these issues in a second draft that would, normally, be their final draft. The result is very rarely a second version that meets both the needs for more fully developed content and corrections in grammar and mechanics.

But this time, I did things differently. Students ended up submitting four versions of the literacy narrative. The first version was their rough draft. When reading and providing feedback on this draft, I focused only on content and organizational issues. The class completed a playlist on Blendspace that focuses on revision and we discussed and practiced some revision in  class with their initial drafts. For the second version of their narrative, I asked students to focus only on addressing the content/organizational issues pointed out in my feedback. Once they had submitted this second version, I marked grammar and mechanical errors and we repeated the same process as we did with revision, this time focusing on the proofreading/editing processes. Once students had submitted a third version that had been proofread/edited, I did a final read-through and addressed any additional issues with content or grammar/mechanics and they submitted their fourth and final version for a summative, holistic assessment.

I found that students did significantly better when it came to both revising and editing their narratives by following this method. What started out, for many, as a skeleton of an essay eventually blossomed into a fully realized piece that was fairly devoid of major errors in grammar/mechanics. In their self-assessments, many students mentioned the positive impact that multiple drafts had on the finished essay and how the process of writing the essay helped them in overcoming some of their fears about writing. I feel that the quality of the essays also proves that poor writing is not necessarily the product of lack of writing ability, but rather a lack of  understanding of the writing process and/or a lack of effort to produce a quality piece of writing, either through apathy, fear of failure and/or criticism, or low self-efficacy beliefs.

Unfortunately, I will not be able to repeat this multiple-draft process with the rest of the pieces that students will be asked to complete this term due to lack of time. My goal now is to figure out a way to make this multiple-draft process workable in the next iteration of the course because I feel the results, both in students’ responses/efforts and the quality of their pieces, are too extraordinary to ignore or neglect due to time constraints. My job as a writing instructor is to find a way to make what works doable. And that’s what I will do.

I would love to hear from those who have identified other methods for effectively teaching the revision and editing processes and those who have found a way to effectively integrate a multiple-draft process into their classes.

Postmortems in the Composition Classroom

The Anatomy Lesson of Dr. Nicolaes Tulp by Rembrandt
The Anatomy Lesson of Dr. Nicolaes Tulp by Rembrandt

I recently ran across an article on grading writing that began by quoting a tweet from a fellow Composition teacher that equated grading a final piece of writing with performing an autopsy on a dead body. I have desperately tried to find the article, but to no avail (if you know the article I’m thinking of or recognize the tweet being referenced, please let me know so that I may give the authors credit).  What puzzled me was that this analogy was meant to have a negative connotation (at least that’s how I read it). I by no means support a summative assessment-only form of grading. I, too, emphasize the process of writing and provide formative assessments that seek to help students to internalize the importance of thoughtful revision and careful proofreading before submitting a “final” version of a piece of writing. And, at one point, I too viewed that final version as a relic to be archived with all of the other finished pieces the student accumulated during the term. But several things have changed for me during the past few semesters: 1) I’ve switched to having students blog instead of submitting traditional word-processed essays; 2) I’ve switched to a portfolio system that allows students to select which pieces they wish to be formally graded, allowing them to revise and edit those pieces before adding them to their portfolio; and 3) I read Lauren Griffin’s “An Open Letter to Writing Instructors from a Motived Student,” which included the following eye-opening (for me) observation:

In many courses, I felt like an overworked employee at an essay factory, producing ten to twelve mediocre and forgettable papers — ones that teachers accepted as final drafts that were, in actuality, first drafts. . . . I wish that all of my instructors had challenged me to produce portfolios with five or six mind-blowing papers instead of valuing quantity over quality.

Griffen, in effect, sums up the kind of traditional method for organizing the First-Year Composition course that I had been told to use as an adjunct and everything wrong with that method: a focus on quantity over quality.

These three things have altered the way I view both the student’s writing process and their writing products, in that I now value both as equally important. For one thing, blogging allows students to view their writing as something alive; they see their readers respond to it and, often, realize that their own views of it change as a result.  Since their blog posts are living things, they can be revised and edited after they’ve been published; much like a garden, a blog needs regular maintenance, from pruning dead links to “growing” or expanding upon a previous post. Secondly, as Griffin argues, the portfolio system encourages writing students to make conscientious changes to pieces. As students read through their work in order to select which pieces to include, they often recognize weaknesses in earlier pieces because they see them in juxtaposition with later pieces. Hence, they begin to recognize their own growth and learning.

I have tried various methods for providing both formative and summative assessments of student writing, including utilizing both anonymous peer review for rough drafts and anonymous peer evaluation of finished products. This past semester, I had great success with having students submit a rough draft to me for feedback and then allowing them to evaluate each others’ finished product (these evaluations were not grade-based and had no impact on the student’s grade). The reason why I made this change was because I have realized that, especially for first-semester composition students, peer review is not as effective as I would like it to be. Even in anonymous, guided peer review, students have difficulty providing constructive criticism on someone else’s draft. Emotions are often involved, either on the reviewer’s end (“I don’t want to hurt anyone’s feelings”) or on the reviewee’s end (“One person said this and another said that, and I’m not sure whose advice to take”). While I think college students need to develop the ability to take criticism, writing is already emotionally fraught for them and the added emotions of peer review seem to make the writing process more, not less, difficult for many students. So, I decided to forego peer review and have students submit their drafts to me for review. While this did require quite a bit of time on my part, I think it paid off in dividends in the students’ final posts. I was able to direct their energies much more effectively than their peers could. I stuck with the minimal marking method and focused on asking questions about the students’ ideas and suggesting areas that needed further development or that seemed off-track. Students responded very enthusiastically to this method and, for the first time ever, I saw students really focusing on revising their writing (rather than the kind of minimalist adding on and editing that often passes for revision with most first-year composition students).

Unlike peer review, students seem particularly adept at evaluating final pieces. I am often amazed at how accurately their evaluations reflect the very things I would have pointed out in my own summative evaluation. I am not sure why this is the case and why they cannot do the same with peer review of rough drafts, but I decided to capitalize on it; since I was investing so much extra time and energy into reading and providing feedback on rough drafts, I completely handed summative assessments over to the students. I did perform a quick read-through of final posts in order to see how much effort the student had put into revising and editing the original draft and I did read through the summative feedback to ensure that student evaluations were accurate. Again, this system was very effective, as it removed the burden of summative assessment off of my shoulders (so that I could focus on helping with the writing process), it gave students multiple assessments of their final product, students were much more honest with each other about weaknesses in their writing, and they genuinely valued their peers’ evaluations and integrated them into their revisions and edits for their portfolio.

Next semester, I would like to add a few more layers of feedback to each piece of writing. I am considering, for example, adding a peer review session back into the process, after my own review of their drafts, to encourage a multiple-draft process. I am hoping that I can model effective feedback methods and encourage students to apply them to their own reviews. I would also like to focus more on the finished product and to integrate a postmortem of that product. I already have students write a reflection on each piece that takes their peers’ summative evaluations into consideration and establishes goals for the next piece of writing. But I would like to encourage students to autopsy their products in a much more explicit way after their emotional attachment to the piece has cooled a bit. While some may view the idea of an autopsy negatively, I see value in the process for FYC students.

Let us consider what, exactly, an autopsy is. It is, foremost, a thorough examination that seeks to determine the cause of death. But an autopsy often reveals much more than the cause of death, including diseases or injuries, both past and recent, that are not directly related to the death but that tell us more about the subject’s life and their relative health. But pathologists are not the only ones who perform autopsies. Game developers also perform postmortems. These postmortems seek to identify strengths and weaknesses in the game and to brainstorm how to improve it in future iterations. I think that having students work in groups to perform postmortems on the pieces they select for their portfolios would be an ideal way to encourage them to both value their final products and consider what they could do to add more life to each to make it an even stronger piece of writing. More than a pathologist, I’d like to encourage my students to become like Victor Frankenstein, seeking to collect the best pieces they can find in order to create something greater than the sum of its parts and imbuing it with life. But, rather than being afraid of and rejecting their creature like Frankenstein, I want them to thoroughly examine it and come to love it, both for its strengths and its faults.

 

Experience Points in the Classroom: Back to the Drawing Board

help_how_leveling_up

One aspect of gamification that I decided to try to integrate into my Fall classes is that of Experience Points (XP), which involves using the same kind of points system that games use to help players feel rewarded for completing certain tasks and to allow them to advance to different levels of experience, providing them with a goal to attain (the highest level) and a visual measurement of their progress toward that goal. I’ll be honest, I don’t think that I integrated XP into my classes very effectively. I will outline how I integrated XP, which aspects of that integration were mistakes, and how I plan to do it better next semester.

Method

First, I determined the maximum amount of XP that a student could earn simply for doing all of the required assignments in the course. Using this as my baseline, I assigned XP to each assignment, with the difficulty of the assignment determining how many XP it would be worth. I also determined which assignments would receive XP based upon completion and which would receive a variable amount of XP based upon quality of work and effort. I created a spreadsheet for students that listed each assignment and the number of XP that could be earned for completing them and included the possible XP earned on the instructions for each assignment.

Next, again using my max. XP as my baseline, I created five levels, with each level corresponding to a letter grade on the 4.0 grade point scale. I gave each level a name that related to our course theme and included a list of the levels and the XP needed to achieve each on the syllabus.

Since I needed an easy way to keep track of XP and to allow students to visually monitor their progress and current level, I decided to use Blackboard’s grade book tool, even though I loathe Blackboard. The most important determining factor for me for using Blackboard was the fact that a colleague who is also interested in games-based learning had created a leaderboard block that can be placed on the homepage of a class’s Blackboard course shell that is tied to the number of points a student has in their Total column in the grade book tool. 

I decided that I did not necessarily want to tie participation XP because I have found that by designating a certain number of comments to make on their peers’ bog posts, I encourage students to see participation and commenting as a hoop to jump through and they will resort to focusing on quantity rather than quality. So, I developed a holistic rubric for participation and made the participation grade 1/3 of a student’s final grade, with their level making up another 1/3, and their final portfolio finishing out their grade. 

Where I went wrong

Firstly, I believe it was a mistake to allow students to see how many XP each assignment was worth. In reality, a player doesn’t begin a game with a spreadsheet of actions and how many points they’ll earn for completing them. A player doesn’t know how many XP they will earn for completing a task or puzzle until it’s completed; this increases the element of uncertainty that, according to Tom Chatfield, is one of the ways in which games reward the brain. This uncertainty motivates players to complete as many tasks and puzzles as they can because they never know what kinds of rewards each will provide. 

Also, while students responded very positively to the leaderboard and found it to be a motivating factor, the fact that their XP were recorded in Blackboard’s grade book meant that, rather then receiving 50 XP for an assignment, for example, a student would receive 50/70 (or whatever amount the maximum number of XP available and earned), transforming XP into a grade. Rather than feeling rewarded for earning 50 XP on an assignment, students began to focus on the fact that they didn’t earn the maximum amount of XP and would often want to talk with me about their “grade” on an assignment. Because XP were situated within a context that students identify as relating to grades and because Blackboard required me to enter a total number of possible points for each assignment, thereby redirecting students’ focus towards the points they didn’t earn, by the middle of the semester the novelty of XP and the leaderboard had quickly worn off and students saw themselves as being at the mercy of their XP rather than seeing themselves as on a quest to earn as many XP as possible.

I believe that this feeling was reinforced by the fact that I had aligned levels with a letter grade, which may have devalued the game for those students who began to fall behind in XP. I cannot be certain that these students gave up because they viewed all of their effort as directly related to a certain grade or if they were already among those students who, statistically, are predetermined to disengage and/or disappear. 

What I’ll do differently next time

For one, I will not advertise the number of XP that can/will be earned for completing assignments and tasks. While I will keep the method I used for determining XP and levels, I will increase the uncertainty for students by keeping that information a secret that will need to be discovered through effort. Levels will remain aligned with XP, but will not aligned with a letter grade.

I will also not use Blackboard to track XP and levels. Instead, I will use Google Drive’s spreadsheet tool to create a system for tracking each student’s XP and create a leaderboard chart based on the column for their total XP. Because XP and levels will no longer be tied to grades, I can give students access to the chart and embed the leaderboard on the class website (I will have students create an alias in order to maintain anonymity). 

I also plan to hide more Easter eggs throughout the game in the form of bonus XP as a way of rewarding desired behavior, such as using the Writing Clinic and significantly revising and/or editing their drafts before publishing them. I’ll let students know that these opportunities exist, but they’ll have to work to figure out which behaviors they’ll be rewarded for.

Because their XP and level will no longer be aligned with a grade, I will need some way to determine how these aspects of a student’s gameplay impact their final grade in the course. I’ve decided that rather than imposing a weighted system for each major component of student behavior, I will use the portfolios that students complete at the end of the semester as a way for students to summarize and evaluate their gameplay, looking at XP and level, participation, and their progress as a writer in order to determine what character class they feel they fall into. Each character class will represent certain attributes and will be aligned with a level of gameplay. While I won’t explicitly link their character class to a grade, I will ask students to argue for the grade that they feel they deserve in the course based on the evidence they use to assign themselves to a character class.

Gamification involves trial and error, just as any instructional method does. While I am dissatisfied with the fact that my FYC students began to focus on grades to an extent that I have been able to avoid for the past few years with a switch to a portfolio assessment system, I still believe that using XP and levels can have a positive impact on student motivation. It is especially helpful to those students who have difficulty transitioning away from a focus on grades and who need a way to visualize their progress and standing in the course. I hope that by integrating the changes outlined above, I can use XP and levels more effectively.

Have you had success integrating XP and levels into your classes? If so, I’d appreciate you sharing your experiences and/or techniques with me and my readers.

Murder, Madness, and Mayhem: A Role-Playing Game for First-Year Compostion

Headless-1

Murder. Madness. Mayhem. What new horrors lurk in the minds of men and women? Real life is scarier and stranger than any fiction. But an intrepid group of investigators are working to make the world a safer, saner place. No matter how old the crime, no matter how elusive the evidence, no matter how powerful those involved, they will leave no stone unturned in their search for the truth. They have no magical weapons with which to assault the dark things of the world. They simply have their wit, courage, and analytical skills to help them do battle with the horrors they face.

This past week I worked on summarizing the results of my attempt to integrate role-play writing into my second-semester FYC class in an article that I plan to submit for the Fall edition of Virtual Education Journal. For me, reflecting on past classes inevitably leads to a desire to begin planning a new (and hopefully better) iteration. Thankfully, I asked the students to provide me with both anonymous constructive feedback on the class and to talk openly with me about how they would redesign the class if they were taking it a second time. Their feedback had two major themes:

  • While they liked Second Life, many students felt it was too clunky and wasn’t integrated into the class in an effective way
  • Many students expressed a desire to have more f2f role-play

As I began to mull over how best to address the two issues, I decided to focus on finding an alternative to Second Life. I was looking for something with a less daunting learning curve that would allow for more challenge and exploration-based interaction. While nothing really presented itself, I did stumble upon a website that changed the direction of my thinking: Epic Words.

Epic Words functions as a portal for an RPG campaign (an ongoing storyline or set of adventures). A GM (game master) can create a campaign for any RPG and add any registered players to the campaign. The site offers several tools in one central location: character blogs, a campaign wiki, a discussion forum, quest logs, a calendar, a page for awarding and tracking XP, and the ability to create loot that can either be awarded by the GM or purchased by the players from merchants. Intrigued, I began to research the concept of campaigns and the various ways that players use tools and sites outside of the game to continue, reinvent, and hack the game.

As  I browsed through the various campaigns on the site, I began to see just how similar the RPG I had designed for my Spring 2013 FYC II class had been to one of the most popular tabletop RPG’s, Call of CthulhuTaking my cue from the game, I have started to sketch out what I hope will be an engaging and immersive RPG experience for next semester’s FYC II class, remixing and hacking the traditional tabletop RPG as needed.

Roles

In Call of Cthulhu, characters are called investigators. Players select the occupation of their character and establish their attributes via dice rolls. Like my class, the nature of the game naturally lends itself to selecting characters who would normally investigate unusual events, such as detectives, psychologists, scholars, etc. I’ll limit my students to occupations that will work with the texts we have in our literature anthology, but will allow them to suggest modifications if they wish. Students will spend some time developing their character’s backstory, creating an avatar for them, and creating a profile for them on Epic Words.

Guilds

While students really enjoyed working in role-based guilds last Spring, many suggested more inter-role interaction in order to consult with experts on other aspects of their “cases.” So, this time around students will have two guilds: a home guild that will be role-based and an expert guild that will be comprised of representatives from all of the roles who will consult with one another as needed.

Quests

The quests will remain the same: students will read assigned “cases” from the literature anthology, discuss and analyze them with their home guild, and select one case to focus on investigating for each quest. They will present their selected case via a blog post, determining what format their character might choose to write about the case in (case notes, interview transcripts, a newspaper/journal article, etc.), and also read and comment (in-character) on other characters’ blog posts.

Boss Level

Last Spring, students selected 1-2 partners to work with to create a penultimate project on one of the term’s cases. While the projects they created were creative, engaging, and demonstrated a deep level of analysis, next term I plan to push the envelope even further and ask students to work in a craft guild to develop and write a piece of interactive fiction about a selected case in which the player has to take on one of the roles from the class game.

Feedback

There will be no grades in the class. For some of my Spring students, this was frustrating and many of them expressed a need to be able to measure their progress and have an idea of just how successfully they were playing the game (aside from the formative feedback they received from me and their peers). Epic Words provides me with several tools that I can use to provide feedback and progress reports to students.

One form of feedback I’ll use to indicate successful completion of quest-related tasks and puzzles is XP (experience points). This has been very successful this term with my FYC I classes. While this term I’ve had to rely on Blackboard’s grade book  to record XP and provide students with a means of measuring their progress via a leader board (more on this in a subsequent post), next term I can use Epic Words, which will allow students to view their XP on the campaign’s XP page.

A second form of feedback Epic Words allows GM’s to create and award is loot, which has allowed a useful hack of Call of Cthulhu’s investigator attributes and skills. Rather than relying on dice roll to determine the attributes of an investigator, I can do so by awarding them loot for demonstrating mastery of various skills, such as research, analysis, creativity, etc. In addition to awarding them skills, I can also award them cash for participation and completing quests. The players can then use this cash to purchase investigative tools, such as flashlights, fingerprint kits, video recorders, and smartphones, from  a merchant (my merchant is called Doyle & Poe Investigative Merchants). Purchasing investigative tools will make their character more powerful. Again, all of a character’s loot can be tracked in Epic Words.

Endgame

How does completing quests and collecting XP and loot translate into a final grade in the course? In order to demonstrate the quality of their work and learning in the course, students will have to submit a portfolio of their game artifacts: their best blog posts; their XP; their skills, cash, and tools; and their forum and wiki contributions. They can then use this portfolio to advocate for the grade they feel they’ve earned in the course.

Design

Research has found that aesthetics can have a significant impact on motivation, immersion, and engagement among game players. I am planning to spend much more time on the visual design of the course than I did last term. Epic Words allows GM’s the add a background image and change the color scheme for campaign sites, as well as add images to pages. Being a fan of all (weird) things Victorian, including the neo-Victorian and steam punk movements, I think pulling design elements from these aesthetic styles will work well with the theme of the game.

Once I’ve finalized the components of the class and the campaign site, I’ll post updates here. I hope that this post inspires you to create your own RPG and/or try Epic Words as a tool for managing your games-based learning. I’d love to hear what you think of my ideas, how you’ve integrated RPG into your own classes, or how my post has inspired you to do so.