Hacking Assessment: Redesigning the Numbers Game

photo credit: davidfg via photo pin cc

In a recent post, I outlined some ideas that I have about integrating principles of game design into the FYC course. As I pointed out, I’m not all-out gung-ho about the idea of the gamification of education. It turns out that many of my reservations about this latest trend in reforming education are shared by game designers themselves. In her post “Everything Is Game Design,” game designer Elizabeth Sampat makes clear that the assumption that any group of practitioners can co-opt and apply the extremely complex and abstract principles at play in a successfully engaging (to some) game to any other domain is over-reaching:

Gamification” assumes all games share the same mechanics, which means everything that’s gamified is basically the same shitty game. Using badges and leaderboards and offering toothless points for clearly-commercial activities isn’t a magic formula that will engage anyone at any time. Demographics are different, behavior is different . . .

These are the same issues with gamifying the classroom that keep me from wholly embracing the concept. For one, the whole point of a game is that it is . . . well, a game. Games are voluntary. As soon as you force someone to join in a game, it stops being a game for them. It becomes a compulsory activity devoid of intrinsic value and all of the extrinsic rewards you can throw at them, while perhaps artificially increasing their motivation to play the game, cannot turn it back into a game, unless it’s in the negative sense. Even when we gamify a class, we’re still making the learning that takes place within that game compulsory and effectively negating any positive characteristics of gaming that we are attempting to channel. And, as Sampat points out, the characteristics that make any game engaging cannot be standardized. What works for one gamer doesn’t work for another. So, in many ways, game designers face the same kinds of issues and challenges that educators face.

Another point that I think has been largely overlooked in this debate is that, for the large majority of students (if not all), school is already a game. We have goals (behavioral or learning objectives), challenges (in-class activities, homework, exams, and standardized tests), and rewards (grades). We’ve got levels (grade levels based on age in K12 and hours-earned status in college) and leaderboards (A/B honor roll in K12 and President’s and Dean’s lists in college). And we have clearly defined roles (teacher as locus of power and expertise, student as powerless and largely silent novitiate). Some students figure out pretty early how to play the game. In college, these are the students whose identity is inextricably intertwined with their grades. “But I’m an ‘A student,'” they insist when faced with anything other than. Other students learn early on how to game the game. These are the students who know how to manipulate the system and those in charge of it and can often be just as successful at winning the game as their overachieving counterparts. But some students never learn how to play the game according to our rules. Others don’t want to play it because they see it for what it is.

Whether we realize it or not, we’re already playing games with our students. And it’s a numbers game. Play the game according to our rules and we’ll reward you with a high GPA and a diploma, with the promise that these things are the badges you need in order to level up to the American Dream. This kind of game is both irrelevant and counterproductive in a culture that is becoming increasingly participatory, rather than competitive, in nature (just read Share or Die: Voices of the Get Lost Generation in the Age of Crisis to get an idea of how important cooperation and collaboration is becoming for those graduating into the current economy). While many educators are fighting to reform the standardized, hierarchical forms of assessment that have been in place since the industrialization of education, until they are successful at effecting a wholesale paradigm shift and not just applying a false facade and calling it reform, we are forced (much like our students) to try to figure out ways to hack the game. As Sampet argues:

Finding the reward structures and the rules that are already in place, and figuring out how to make them more effective, is the key to making life better for everyone— not adding an additional layer of uninspiring mechanics that push us to engage with mechanics that already suck.

Just as games are not one-size-fits-all, assessment shouldn’t be one-size-fits-all, neither in terms of standardized criteria applied to all students nor evaluative formats used for all courses/disciplines. Just as each course has its own unique set of learning objectives, each course should have a different method for assessing how students go about achieving those objectives. I think it important to explore various assessment methods in an effort to find which is the most effective for a particular course. For example, I have found that a portfolio method is exceptionally well-suited for my composition courses, as it allows for the abstract nature of the writing process and the subjectiveness that characterizes the act of evaluating and valuing a piece of writing. But in trying to incorporate a portfolio system into my speech courses (both an introductory oral communication class and an advanced argumentation and debate class), I have had less success, though for different reasons (perhaps due to the differences among the students: freshman and upper level secondary-education majors, respectively). As much as the portfolio method places value on each student’s individual learning needs, goals, and achievements, within the current grades-based system, students in certain courses need to be able to visualize their learning at both a qualitative and quantitative level. So, what are the alternatives?

Peer Assessment
One option that is gaining ground is peer assessment. Cathy Davidson has successfully explored this method in her “This Is Your Brain on the Internet” class (read “How to Crowdsource Grading” for her description of the process and the thought-provoking debate that followed and “How to Crowdsource Grading: A Report Card” for an overview of her students’ responses to the method). Many MOOCs utilize peer assessment out of necessity. According to Debbie Morrison, within the MOOC environment, peer assessment results in an enhanced learning experience for the student, as grading their peers’ work requires a deeper engagement with course content.

I’ve utilized peer assessment in both of my speech classes to varying degrees and with varying levels of success. In my introductory speech class, the students work together at the beginning of the term to develop a checklist for an effective speech (I don’t use rubrics because, in my experience, they become just another hierarchical form of grading that allows students to retain many of the gaming habits they adopted in K12). They do this by watching several speeches on YouTube and creating individual lists of do’s and don’ts, which we then collate into a master list. For each speech, students are evaluated by five randomly selected anonymous peers, who use the checklist to assess the speech. The students are also filmed and they must use both the video and their peers’ checklists to compose an assessment of their speech that they post to an e-portfolio, along with all artifacts associated with the speech (outlines, bibliographies, slideshows, photos of visual aids, the video of the speech, etc.). For this particular class, I have found that a combination of self and peer assessment has been much more effective than a solely self-based assessment (which tended to be superficial) or even an instructor-based assessment (in which students received only one assessment, as opposed to five, and tended to focus more on improving their “grade” than becoming a more effective speaker). With the peer assessment method, students’ speeches are being evaluated by their audience and their focus becomes oriented towards improving their audience’s response to subsequent speeches.

I have tried this kind of peer assessment in my debate class with far less success. For one, the class is much smaller, and consists, for the most part, of a cohort of sophomore and junior-level secondary education majors. These students tend to be very cliquish and ironically conservative in terms of the practices they expect in the class; they tend to be “A-gamers” obsessed with acing the course and uncomfortable with the level of abstractness and improvisation involved in debate. As a result, they tend to assess their peers over-generously and resist critiquing one another (one class even admitted to giving each other positive assessments across the board because they didn’t want to “hurt someone’s grade”). They look to me as the expert, so their portfolio reflections tend to be focused on flattering me and the course and highlighting aspects of their performances from my point of view (“If I were the instructor, I would give this speech a [insert grade here]”). Despite my best efforts, these students are resistant to assessment formats that are not instructor-based. So what’s a disruptive pedagogue to do?

Contract Grading
While I was at first dismissive of contract grading based on the distaste I harbor for the artificially hierarchical nature of any type of grades-based assessment (and the name’s implications of a kind of capitalistic supply and demand relationship between student and teacher), I have become less dismissive of the method in terms of its ability to bridge the gap between my students’ need for a quantitative value to be placed on their learning and my own objective of encouraging them to recognize and become complicit in the qualitative value of that learning.

For one, I’m hoping that it will eliminate the specter of grades that haunts the course by directly addressing the students’ anxiety regarding their status in a course that has no exams or other easily quantifiable activities. Students will decide what grade they wish to work towards and will have a specific, objective set of criteria that they must achieve in order to earn that grade (yes, I know this sounds just like a syllabus with a traditional grading schema, but contract grading makes the implicit aspects of the traditional schema explicit and, in many ways, mimics the game design principle of starting at zero and gaining points as you go). Once the question of grades is out of the way, perhaps the students will be more willing to focus on learning and improving.

Secondly, contract grading requires student input in regards to the challenges that must be met in order to level-up (yes, I know I’m wading back into gaming territory, but, as I’ve argued, our goal should be figuring out what works for a particular course and cohort of students rather than a wholesale dismissal or acceptance of any one method or theory). Often, in order to earn an A or a B, students must complete additional learning tasks, sometimes choosing between several options, which they can be invited to develop. This aspect of contract grading is the one that I find most promising in terms of encouraging student investment in the learning environment. While I have long preached to students that, in the words of Lennon and McCartney, “in the end, the love you take is equal to the love you make,” contract grading makes student-centered initiative an explicitly integral component of the course.

Thirdly, contract grading will allow me to both address the students’ insistence that I fulfill the role of expert assessor and my wish for them to fulfill the role of deliberate and reflective practitioner. Different grades require different levels of mastery, so students who contract for a certain grade must revise and/or re-attempt assignments that don’t demonstrate mastery. While my debate students can’t re-do a live debate, they can complete a video re-enactment that improves upon their live performance or record a play-by-play self-critique using Voice Thread or screencasting software. In addition, some of the optional assignments can require peer or self-assessment or other types of reflective learning practices.

While I’m not completely comfortable with contract grading (just as I am not completely comfortable with gamification), I also recognize that other assessment methods are not working for my upperclassman and, as a result, are interfering with my efforts to push them beyond a superficial engagement with their learning in the course. I believe firmly that we must recognize our students’ needs, values, and histories; but we can’t pick and choose which of those we take into consideration when designing their learning environments. Sampet makes a point that I think is important for us to keep in mind in the process:

The core principle to remember is that game design is everywhere. Instead of trying to stick a crappy, half-formed game onto real life, the real challenge— the one that’s tough, the one that will bring the greatest results— is to fix the bad game design that’s all around us.

Students won’t be open to assessment that values quality over quantity or process over product until we recognize that our current assessment paradigm is a badly designed game that needs to be torn down and redesigned. Sampet suggests two questions to ask when considering whether or not something is badly designed:

  • What’s supposed to be the goal here?
  • Is this experience set up to help or hinder my ability to reach that goal?
I’m game.
Resources on Contract Grading
These are the sources that I consulted to help me to better understand the possibilities afforded by contract grading:


Gaming the First-Year Composition Course

In my last post, I addressed the idea of disrupting the First-Year Composition course. One of those disruptive pedagogies that I’ve been monitoring for some time is gamification. I don’t like jumping on any pedagogical bandwagon until I’ve had some time to observe it from afar for a while and reflect on how it fits within my own teaching philosophy and practices. I’ve been doing so with the concept of gamification for almost two years now and up until recently was still uncertain about how I felt about it and how it would benefit my FYC students, if at all. This post is my attempt to clarify some of my initial conclusions on how game theory might be used to help make the FYC experience more engaging for students.

[Disclaimer: This post will not seek to debate gamification’s merits and/or deficiencies. I have mixed feelings about the application of gaming to teaching, some of which I will address in this post. It’s also important to differentiate gamification from game-based learning–the direct use of games and game creation within the classroom. I’m more concerned with how we can use the philosophy of game design to guide our pedagogical practices.]

For me, my own ideas about how gaming philosophy can be integrated into the FYC course were solidified as I watched this TEDx Talk by Paul Anderson, in which he outlines why and how he gamified his science classes:

Recently, this same video was the focus of a post by Adam Renfro on the Getting Smart blog. The post does an excellent job of breaking down and explaining the elements of gamification and how they can be applied to any class. As I read the post, I became increasingly aware of how much I am already applying the principles of gamification to my FYC classes. But the post and video inspired me to consider other aspects of my course that could be gamified to create a more immersive and disruptive experience, so I sat down with pen and paper and, using the outline Renfro provides in his post, did some brainstorming. Here’s what I came up with:

The Story
For me, the story is always supplied by a course theme. One semester it was how education is used as political currency and the lengths that people will go to to get an education; another semester it was the freshman year experience; next semester it will be the purposes, strengths, and shortcomings of universities in the 21st century. I use the course theme to help me select the nonfiction books that we read together as a class and to provide a focus for the students’ self-selected reading, but the students write the “story” themselves, choosing which of the infinite plot lines within our theme they wish to pick up and develop in their writing (in much the same way that “choose your own adventure” books work).

Clear Goals
As Renfro points out, in gaming, goals are concise, specific, and clear (no behavioral objective jargon or Bloom’s taxonomy verbs to muddy up what needs to be done or why). While I’ll still have to use the course objectives provided by my department as written (for some esoteric and, more than likely, bureaucratic reason), I’ll spend some time explaining those goals in plainer language on the course website and I’ll certainly begin to utilize the kinds of clear goals used in gaming when designing the assignments and tasks for the course. [As a rather disturbing anecdote, one semester I asked my students to re-write the course objectives from the syllabus in their own words and explain what the objectives meant in terms of what they needed to learn to do; not a single student could do so, even after looking up all of the unfamiliar words in a dictionary.]

Challenges
The most obvious challenges to establish in an FYC course are the writing assignments. For my students that means creating and maintaining a blog where they publish all of their writing for the class (the “story” they choose to tell about our theme) and reading and commenting on their peers’ blog posts. It will also mean using the skills they develop over the course of the term to solve a relevant problem for our university and its goal to become a 21st century learning environment (I’ve addressed this in a previous post).

Reading, as Anderson acknowledges in his TEDx Talk, is also a challenge for many students. Next term, my students will crowdsource the reading of our class book by collectively annotating it using Google Docs. This challenge works in tandem with two other challenges that I will establish: improving their digital literacy skills (they’ll be annotating Howard Rheingold’s Net Smart) and building a Collaborative Learning Network. Part of the students’ objective in annotating the book is to create their own challenges for integrating the skills discussed in the book into the class. This type of self-authored challenge opportunity is one aspect of gaming that is becoming more popular (my 9 year-old son, who is an avid Lego architect and gamer, revels in games that require him to build his own gaming environments).

Image courtesy of Technorati

Competition
For me, this is one of more problematic aspects of game theory in terms of its pedagogical applications. I recognize that competition can be healthy, I’m just not convinced that the classroom is a context within which that is the case. If students decide, on their own, to compete with their peers to achieve a certain number of “likes,” “+1’s,” or shares, then that is fine, but I’m not comfortable creating forced competition.

Defining the Roles
Since my FYC classes are hybrid, I require that students create an avatar to use in all of our virtual learning environments. I’ve streamlined this as much as possible by using all Google apps for our virtual class work. Students create a Google account during the first week of class and complete a Google profile page with an image of their choosing. They use Blogger for their blogs, Google+ for virtual interaction, and Google Docs for collaborative writing, so their interactions are automatically associated with their avatar. For their first blog post they select a skill or passion to share with their peers as way of introduction. This assignment usually reveals some gurus and go-to’s for various aspects of the course (this term, for instance, I had a tech geek, a journalism major, and a cheerleader, all skills highly valued in an FYC course for various reasons). I encourage students to seek out peers who posses the domain skills that they are in need of if I’m not available or skilled enough to help them, and I encourage students to use their individual skills and personality traits to build and support a collaborative community in both the physical and virtual learning environments.

Equipment
Rather than relying solely on a writing handbook, I’ve begun compiling videos, handouts, and web pages that I can direct students to when they need additional guidance. Last term I experimented with not using a handbook at all and, instead, created a wiki of writing resources. For each writing concept, I tried to provide as many different varieties of resources as possible: at least one video; a concise overview or outline of the concept; a longer, more detailed web page; at least one source that provided examples; and a PDF handout or graphic that they could print out and keep handy. Many students responded enthusiastically to this method and the resources themselves and I received overwhelmingly positive feedback regarding the wiki when I polled students on the most effective aspects of the course. This term, I plan to organize these materials into different lessons on Mentor Mob and invite students to add to them (as Renfro points out, the challenge is increased for the students when you allow them to create and use their own equipment).

Scaffolding
Renfro warns that giving all course materials out at once is confusing for some students. This, of course, runs counter to what many consider “best practice” in hybrid and online teaching, which holds that everything should be front-loaded so that your expectations and the course requirements are clear and students have access to the materials so that they can work ahead if they wish. In my experience this has had two results: for weaker students, it is overwhelming and they tend to take an “if I ignore it, it will go away” approach to accessing and reading materials; for stronger students with type-A personalities, this creates anxiety as they constantly try to stay ahead of the game and often miss out on what’s happening in the moment. Next term, rather than uploading all of the writing assignments to a static page on the class’s WordPress site, I plan to post assignments to the blog as I feel they need to be on students’ radars; this has the added advantage of providing a central location for students to post questions and comments on the assignment and for me to answer them.

Badges
Right now, I’m still observing and reflecting on the badge system. Students are already familiar with social media’s voting systems, so I will encourage them to use the existing systems to promote and reward each others’ work.

Level Up
I already provide a kind of leveling up system via students’ self-assessments of their work and the formative feedback that I provide on these assessments (see my post on deliberate practice). I ask students to identify the weaknesses in a piece of writing and to work on improving those areas in their subsequent pieces. Once the student feels that they have developed those areas sufficiently, then they must identify new areas to address, essentially leveling up to a new set of criteria. At this point I haven’t established a hierarchy of levels because I am mainly concerned with getting students engaged with the act of writing and I don’t want to discourage their own assessment of their writing by imposing my own rules about which weaknesses to tackle first. While I might value sentence construction more than paragraph organization, for example, the student might find it less daunting to better their paragraph organization than their sentence constructions. (I’ve found that students generally know their weaknesses and have a good sense of which ones can easily be corrected with some resources and a little more effort and which ones will require intensive, and likely frustratingly difficult, work). I’m not sure if I want to enforce a hierarchy of levels or continue allowing the student to determine at what level they wish to work at any given time. The ability to select different levels of difficulty may be a more important gaming principle to apply to the FYC course than scaffolding of skills.

Leader-boards
Because this aspect of gaming is directly tied to competition, it’s problematic for me and I’m not willing to advocate it.

Flipping for Individualization
Like gamification, flipping the classroom is a hotly debated pedagogical disruption right now. I’m not so much interested in debating it here as thinking about what aspects of it make sense and can be used effectively. English teachers have basically been flipping our classes since time began, so it’s a moot point for FYC, as far as I’m concerned. The aspects of the flipped class that I think teachers of writing need to pay attention to is how it allows students to work at their own pace and how it allows us to individualize their instructional needs. I’ve already discussed how I encourage students to work at self-selected levels by assessing their writing, setting goals for improvement, then monitoring their progress with the help of my formative feedback. When this type of self-paced goal-setting is combined with access to a variety of resources that you have gathered or created and made available using a wiki or a tool like Mentor Mob, this gives the student the power to shape the course to meet their individual learning needs. Students don’t waste time on skills they already posses, they don’t have to spend a week on a skill if they only need a day, and they can spend two weeks (or three or four) on a skill that they couldn’t master in one.

Failing
I’ve already addressed failure in a previous post. I truly believe that one of the most effective ways to eliminate students’ fear of failure is by doing away with grades. Until then, the portfolio system is the next best thing in terms of removing both anxieties surrounding individual assignments and the overarching stigma of failure. For each piece of formal writing, my students receive formative feedback from me but no grade. I encourage them to view each piece of writing as deliberate writing practice, the same kind of practice that gamers are free to enjoy without anxiety or stigma if they fail to level up. At the end of the term, the students select which pieces of writing they want me to use to determine their grade for the course and provide me with detailed input on why they selected each piece and what they think it demonstrates about their writing abilities. If at any point a student is uncertain of where they stand in terms of their progress in the course, I will discuss their concerns, but try to steer clear of situating the discussion within the context of grades or points.

Walkthroughs and Cheat Codes
Two aspects of gaming not mentioned by Renfro are walkthroughs and cheat codes. Walkthroughs demonstrate step-by-step instructions for navigating a game environment, while cheat codes are glitches that allow players to cheat the game by accessing hidden objects, shortcuts, or locked characters. Both are deployed to make the game easier or to give the player an advantage over the game. One way that I’ve been experimenting with walkthroughs this term is by using one of the students’ pieces as a model for effective writing, then conducting a paragraph-by-paragraph walkthrough of the piece with me recording our discussion and marking up the text using the Show Me iPad app; once I post the link to the video of our walkthrough, students can revisit and watch it if they feel the need to do so. Another possible way of encouraging the use of walkthroughs and cheat codes in the FYC course is the use of peer instruction. As outlined in the Harvard Magazine article “Twilight of the Lecture” and demonstrated in this video, peer instruction harnesses the collective brainpower of small groups:

By identifying muddy points and misconceptions, then allowing students to discuss and work them out in small groups, peer instruction applies the same methods used by gamers as they crowdsource to share tactics and problem-solve how to game the game.

These are some ways that I think gamification can be applied to the FYC course. Below are a few resources that have helped me to better understand gamification and the pedagogical implications it holds. I’ve tried to provide a balance between the pros and cons of gamification; however, this is by no means an exhaustive list and I welcome any additions you can make to it or any thoughts/experiences that you wish to share about how the principles of game design should or should not be applied to the FYC classroom.

“What Video Games Have to Teach Us about Learning” by Jessie Chuang

“How to Hack into the Joy of Gaming” by Susan Lucille Davis

“Gamification in Education: What, How, Why Bother?” by Joey J. Lee & Jessica Hammer

“Jury Out on Zamzee, Other Forms of ‘Gamification'” by James Temple

“Kathy Sierra on Gamification in Education” by Larry Ferlazzo

“My Practical Criticism of Gamification: Why Not Do Better?” by Ishai Barnoy

“Gamification: Bring Game Mechanics into Non-Gaming Environments” by Adam Renfro

Building a Better Blogging Assignment Redux

photo credit: Mike Licht, NotionsCapital.com via photo pin cc

One of the sessions at last week’s THATCamp dealt with the issue of designing a better model of student blogging. You can view my Storify of the session here.

I thought that I would add some of my own ideas and discuss how I address some of the issues raised during the session (since, unfortunately, I couldn’t be there).

As noted on the session’s Google Doc, a major problem with requiring students to blog is that the large majority of them are unfamiliar with blogs, so we need to identify effective methods for acculturating them to the genre. Since I’m an advocate of immersive learning, I’ve found that many students begin to “get” blogging by spending a good deal of time actually doing it. But I’ve developed a few orientation assignments that help them get off to a good start.

  • Require students to locate, deconstruct, assess, and subscribe to blogs on topics that interest them: As homework during the first week of class, I have students locate several blogs on a topic that they’re interested in. They pick the best three and subscribe to them. While exploring blogs on their topic, they create a list of criteria for an effective blog. We use a class meeting to collate their criteria into a master list that they can then use as a checklist for their own blogs. Next term I’m planning to expand this assignment by having students work together to deconstruct a blog.
  • Teach them how to comment: This is something that I still struggle with. I provide students with several resources on commenting, including those mentioned at the session; nonetheless, many of them provide largely superficial comments. Next term I plan to have students read and assess comments on the blogs they’ve subscribed to and add their own comments. Similarly to the assignment above, students will work together to establish criteria for effective commenting.

A second, and equally important issue, is the logistics of blog management, both for yourself and the students: controlling pacing (so that you don’t have to deal with an influx of posts and comments at the last minute), encouraging engagement with the blogs (both their own and their peers’), and assessing the blogs.

  • Establish submission guidelines (and stick to them): I establish strict deadlines for post submissions and stick to them from the very first post. I generally make the deadline the night before class in the case of totally face-to-face courses. For my hybrid courses, the deadline is on the day that we do not meet. Either way, I set the deadline for a time well before I and other students need to access the blogs.
  • Encourage engagement with peers’ blogs: I require that students subscribe to each others’ blogs and read and comment on a certain number of them each week. I’ve tried to encourage more depth to their comments by staggering the due dates for posts and comments (generally they have 12-24 hours after the blog post deadline to read and respond to peers’ posts). I’ve had even better success this past term with combining this with rotating students’ roles between posters and readers/commenters. This allows them to fully focus on and engage in their role. This method requires reducing the number and frequency of posts for each student, but I think that the pay-off will be worth it, especially by placing as much emphasis on their comments on others’ blogs as on their own blog posts (which means that I’ll have to invest more time into assessing their comments somehow).
  • Make the blogs an integral component of the course: I try to immerse students in their blogs as much as possible because I’ve found that the more they blog, the better bloggers they become. I now require that all of their writing be done on their blog and I ask them to blog and comment on blogs as frequently as possible (at least once a week). I think that it’s a major mistake to have students blog but then not integrate the blogs into the classroom interactions in some way; this encourages students to view the blogs as secondary to the other class work. In my literature courses, the students’ blogs become the fulcrum for the class discussions. I encourage students to pick the most thought-provoking for us to look at together in class. In my FYC courses, I pick one model post each week for us to critique as a class, asking students to assess the post in small groups, looking for reasons why I selected the post as being a good model. Since the class uses Google+ as a virtual learning space, I also “plus 1” those posts that are especially thought-provoking, well written, and/or visually appealing (I encourage students to do this, as well); this provides students with almost instantaneous feedback and encourages those who might not have read and/or commented on the posts to do so. This also results in a type of gamification of the blogs, as some students begin to work to earn “plus 1’s” from me and their peers. Next term, I plan to also encourage students to use other social media to promote and “like” their peers’ posts.
  • Involve students in the assessment of their blogs: In a previous post, I outlined how I require students to self-assess their writing. I have been happy with the way I’ve asked students to create a portfolio of their blog posts to submit to me at the end of term, rather than assigning a grade to each individual blog post (I’ve tried to eliminate traditional grades as much as possible in my classes). Normally, I have students do this via a final assessment form that they fill in and submit to me via email, hyperlinking to specific posts that they want to include in their assessment, and discussing in detail why they selected them and how they demonstrate what they’ve learned about writing. But I’m considering remixing Mark Sample’s idea of a blog audit; I think that making their reflections public on their blogs will encourage an even deeper consideration of who they are as writers and what they’ve done as bloggers over the course of the term, mirroring the way that many bloggers use their blogs as reflective spaces. I also like his idea of having students revisit and revise some of their old posts, which is something I used to encourage students to do with their writing before I switched to blogs, and would like to re-incorporate into their portfolio creation.
  • Utilize formative and peer assessment: This is still something that I’m tweaking. So far, I’ve found my method for providing formative assessment effective (and students have indicated the same). What I haven’t been able to integrate as effectively is peer assessment. I would love to use a badge system, like Mozilla’s Open Badges, but I haven’t had the time to figure out the best way to do so (or if it’s even possible, since I don’t know how to code or if it’s necessary to know how to do so to use the program, two issues I’m hoping to remedy soon). In the meantime, I’ll encourage the use of readily available social media feedback systems such as Facebook’s “like” and Google’s “plus 1” buttons.

A third issue that seems to have been prevalent during the session is that of how to allow for disruption and alternatives within the blogging domain.

  • Allow/encourage alternative uses for blogs: Since I require that students publish all of their writings for the class to their blog, this means that sometimes their blog posts contain nontraditional material (although I always try to help students understand that, with the advent of photoblogs, vlogs, and podcasting, there is no longer such a thing as traditional blog content). For example, this term I’m requiring my FYC students to use Storify to create their annotated bibliographies and then embed their stories into their blogs for comment by me and their peers. Last term, my students participated in DS 106, which meant that their blogs became populated with memes, mashups, animated gifs, and sound clouds.
  • Disrupt the digital environment: Interestingly enough, as participants were discussing Mills Kelly’s ideas about disruptive pedagogies and then subsequently considering ways to disrupt student blogging, I was blogging about Paul Fyfe’s theory of teaching naked and considering how to disrupt the digital environments within which I ask my students to work. One idea that I blogged about that serendipitously showed up on the blogging session Google Doc is that of requiring students to engage with and use their blog posts in non-digital ways. I think that this is an aspect of student blogging that needs more attention and I hope that a conversation can develop around it.

These are just a few of the blogging methods that I have found effective and, as indicated, I’m still working at improving some of them. I encourage those who require their students to blog or who are thinking of doing so to help continue the conversation here, on my Storify of the THATCamp session, on Mark Sample’s THATCamp blog post, or on Twitter (use the #thatcamp hashtag).

The Role of Self-Assessment in Deliberate Practice

photo credit: dkuropatwa via photo pin cc

In my last post, I discussed the need for students to engage in deliberate practice. I think that this is especially true in First-Year Composition courses. For one thing, I’m not sure that we can really teach students how to write. I think we can give them some best practices to follow and show them models of good writing, but writing is one of those skills that you can only learn by doing. And writing, especially academic writing, is a complex skill that takes years to develop. And I only have 14 weeks ( or in the case of my current Summer short-term class, eight weeks).

The other problem that we face in the FYC classroom is the fact that our students come to us with such varied abilities and backgrounds in writing instruction. Some have had little instruction in writing or, if they have, it was poor instruction because they struggle to write coherent sentences and put them together in a logically-organized paragraph. Some have had intense instruction in a very structured form of writing (the old five paragraph, 5-7 sentences per paragraph, keyhole style of essay) that works well on standardized exams but does not allow for the varied disciplinary styles that they will be asked to tackle in college. Some have had their heads filled with a lot of bullshit do’s and don’ts: don’t start a sentence with and or but; don’t use the first-person pronoun; always start your essay with a catchy hook, preferably something that sounds cosmically philosophical; always place your thesis at the very end of your introductory paragraph. I always have my FYC students read a chapter from Surviving Freshman Composition by Scott Edelstein called “The Truth about Freshman Composition” because it does an excellent job of explaining the differences between the kind of writing instruction they received in high school and the kind that they will (hopefully) receive in college and it also dispels a lot of the writing myths that they almost certainly have been taught. Students are always surprised and sometimes even angry that so much of what they were taught in high school has not prepared them for writing in college and, in some cases, was just plain wrong. So I spend quite a bit of time forcing students to unlearn bad writing habits and learn new ones, only the new ones I ask them to learn deal less with how to write and more with how to think about what they’re writing and how to assess how well it accomplishes their purposes. I provide them with lots and lots of chances to deliberately practice writing an academic essay, and with each practice, I ask them to assess what went well and what didn’t go so well and what they need to focus on improving on in their next practice. Here’s my method for doing so.

Even though my students will eventually publish their essays on their blogs, I have them type them up in Word or Open Office first. For one thing, Word will catch some of the more blatant typos and grammar errors that wouldn’t be caught if they were composing within the Blogger dashboard. And if I happen to be using peer review that semester (some semesters I do, some I don’t), I always have them do so from hard copies, which are much easier to print out and read from Word. Some of the newer versions of Word even have a blogging template that will allow students to easily type their posts up in Word and then publish them to their blog.

Another pro of having students initially type their blog posts in Word is that I can have the students highlight and annotate their essays using Word’s commenting tool (I’ve tried Google Docs, but the commenting tool does not allow for the kind of detail that I need when providing my own annotations). I ask students to highlight and comment on any parts of the essay that they have questions/concerns about and to use the commenting tool to communicate their questions/concerns to me so that I can address them. Students rarely take me up on this offer, but some do, so I continue to encourage them to do so. But the real purpose of the Word version of their post is provide them the space to answer five questions that require them to assess the essay. The questions vary from semester to semester, depending on if I’m using peer review or if I’m focusing more on writing process or revision, but they always have the same goal: to encourage the student to reflect on their writing using their own judgement and valuation, rather than waiting for me to pass judgement on the piece’s value. Here’s the five questions I had students answer last semester:

  1. What do you think is working well in this blog post?
  2. What do you think is not working well in this blog post?
  3. What challenged you the most about this blog post and how did you overcome the challenge? If you didn’t overcome it, how will you deal with this challenge the next time?
  4. How successful were you in addressing the weakness that you and/or I identified in you last blog post?
  5. Do you have any questions for me?

The three questions that, to me, are the most essential are 1 (because I think it’s just as important that they be able to recognize strengths as weaknesses), 2, and 4.

After reading and annotating the student’s essay using Richard Haswell’s minimal marking method, I then focus my feedback on their answers to these questions. Sometimes, in the case of a student who is not adept at assessing their own writing, my feedback focuses on correcting their misconceptions about their writing. This past semester, for example, I had a student who was extremely resistant to self-assessment and refused to admit that there were weaknesses in her writing, so I spent my initial feedback efforts in trying to convince her of the necessity of taking an honest look at her writing; eventually, my frustration with her resistance got the better of me and I dedicated all of my feedback to listing all of the weaknesses in her essay (needless to say, her response was less than positive; she complained to the Department Chair about how mean and uncaring I was because I was constantly criticizing her writing). For those students who are more open to self-assessment and are, consequently, much better at honestly evaluating their writing, my feedback efforts are focused on providing tips and links to resources that will help them address their weaknesses. When one student expressed a dissatisfaction with her rough drafts, I suggested that she read Anne Lemott’s “Shitty First Drafts.” On the next self-assessment, the student thanked me for suggesting that she read it and said that it helped her out tremendously. She then suggested it to another student in her comments on a blog post in which they expressed frustration with the invention stage of the writing process.

Not all students act on my recommendations and even fewer pass them along to their peers, but at least their self-assessments provide a dialogue that is not encouraged in traditional, instructor-centered summative assessment models. And this dialogue continues throughout the semester, as students use their previous self-assessments and my feedback on them to answer the next. This dialogue culminates in the writing portfolio that students submit at the end of the term. In putting together their portfolios, students have a semester’s worth of assessments that provide a narrative map of their progress as writers. They can use these narratives to select representative pieces of writing and write their final self-assessment. But it’s only final in terms of that particular class. For the portfolio, I ask them to identify aspects of their writing that they still see as weaknesses and to discuss how they plan to continue to deliberately practice at eliminating those weaknesses from their writing.

photo credit: giulia.forsythe via photo pin cc

Because they have been conditioned by their K12 education to see the teacher as the sole authority in evaluating and valuing their learning, some students need guidance in assessing their own writing and a small minority will be resistant to doing so. But for those students who are willing to learn how to do so, self-assessment can mean much more productive practice and, based on my observations, results in more meaningful learning than that experienced by students who depend solely on their instructor’s summative evaluations. This past semester, I asked my two FYC classes to anonymously respond to a midterm course evaluation. One of the questions asked them what aspect of the course had helped them to improve their writing the most, and the majority of students indicated that the self-assessments had been one of the most helpful aspects of the course (second only to blogging). Here’s a few examples of students’ feedback on the self-assessments:

  • Having to specifically address issues in our writing through our [self-assessments] has helped me out immensely.
  • The instructor commenting on my writing and telling me how I can improve. FEEDBACK from the instructor helps a lot.
  • I really like how helpful you have been. I really like the [self-assessments] we get back each week.
  • I love the [self-assessments]. They help me. ALOT.

As an instructor who often struggles with doubts about the impact I am having on my students’ writing, that ALOT, though misspelled, really means A LOT.

By the way, if you’re wondering about why I needed to change the wording on the feedback, I don’t call the question sets self-assessments. I call them process memos or revision memos (again, depending on the focus of the course and the questions I’m asking them to answer). So, my students may not even realize that they’re engaging in the grading process (I don’t like the term grade, but that’s their only frame of reference and that’s what I’m ultimately required to do to their writing). And I’m not sure that it would be a good idea to muddy the water by telling them this. I don’t want them to start saying things like “I’d rate this essay at a B,” like their writing is an egg that met certain interior and exterior qualities at the time it was packaged.

My student self-assessment system is, as is everything I do as an instructor, a work in progress. There may be better questions that I can ask. And I’m not sure I’m very good at  teaching students how to evaluate their own writing. I’m interested in how others ask their students to assess their own learning and how they guide them in doing so. Please share your tips and experiences. How can we encourage students to assess themselves and be less dependent upon us as arbiters of their learning?