Postmortems in the Composition Classroom

The Anatomy Lesson of Dr. Nicolaes Tulp by Rembrandt
The Anatomy Lesson of Dr. Nicolaes Tulp by Rembrandt

I recently ran across an article on grading writing that began by quoting a tweet from a fellow Composition teacher that equated grading a final piece of writing with performing an autopsy on a dead body. I have desperately tried to find the article, but to no avail (if you know the article I’m thinking of or recognize the tweet being referenced, please let me know so that I may give the authors credit).  What puzzled me was that this analogy was meant to have a negative connotation (at least that’s how I read it). I by no means support a summative assessment-only form of grading. I, too, emphasize the process of writing and provide formative assessments that seek to help students to internalize the importance of thoughtful revision and careful proofreading before submitting a “final” version of a piece of writing. And, at one point, I too viewed that final version as a relic to be archived with all of the other finished pieces the student accumulated during the term. But several things have changed for me during the past few semesters: 1) I’ve switched to having students blog instead of submitting traditional word-processed essays; 2) I’ve switched to a portfolio system that allows students to select which pieces they wish to be formally graded, allowing them to revise and edit those pieces before adding them to their portfolio; and 3) I read Lauren Griffin’s “An Open Letter to Writing Instructors from a Motived Student,” which included the following eye-opening (for me) observation:

In many courses, I felt like an overworked employee at an essay factory, producing ten to twelve mediocre and forgettable papers — ones that teachers accepted as final drafts that were, in actuality, first drafts. . . . I wish that all of my instructors had challenged me to produce portfolios with five or six mind-blowing papers instead of valuing quantity over quality.

Griffen, in effect, sums up the kind of traditional method for organizing the First-Year Composition course that I had been told to use as an adjunct and everything wrong with that method: a focus on quantity over quality.

These three things have altered the way I view both the student’s writing process and their writing products, in that I now value both as equally important. For one thing, blogging allows students to view their writing as something alive; they see their readers respond to it and, often, realize that their own views of it change as a result.  Since their blog posts are living things, they can be revised and edited after they’ve been published; much like a garden, a blog needs regular maintenance, from pruning dead links to “growing” or expanding upon a previous post. Secondly, as Griffin argues, the portfolio system encourages writing students to make conscientious changes to pieces. As students read through their work in order to select which pieces to include, they often recognize weaknesses in earlier pieces because they see them in juxtaposition with later pieces. Hence, they begin to recognize their own growth and learning.

I have tried various methods for providing both formative and summative assessments of student writing, including utilizing both anonymous peer review for rough drafts and anonymous peer evaluation of finished products. This past semester, I had great success with having students submit a rough draft to me for feedback and then allowing them to evaluate each others’ finished product (these evaluations were not grade-based and had no impact on the student’s grade). The reason why I made this change was because I have realized that, especially for first-semester composition students, peer review is not as effective as I would like it to be. Even in anonymous, guided peer review, students have difficulty providing constructive criticism on someone else’s draft. Emotions are often involved, either on the reviewer’s end (“I don’t want to hurt anyone’s feelings”) or on the reviewee’s end (“One person said this and another said that, and I’m not sure whose advice to take”). While I think college students need to develop the ability to take criticism, writing is already emotionally fraught for them and the added emotions of peer review seem to make the writing process more, not less, difficult for many students. So, I decided to forego peer review and have students submit their drafts to me for review. While this did require quite a bit of time on my part, I think it paid off in dividends in the students’ final posts. I was able to direct their energies much more effectively than their peers could. I stuck with the minimal marking method and focused on asking questions about the students’ ideas and suggesting areas that needed further development or that seemed off-track. Students responded very enthusiastically to this method and, for the first time ever, I saw students really focusing on revising their writing (rather than the kind of minimalist adding on and editing that often passes for revision with most first-year composition students).

Unlike peer review, students seem particularly adept at evaluating final pieces. I am often amazed at how accurately their evaluations reflect the very things I would have pointed out in my own summative evaluation. I am not sure why this is the case and why they cannot do the same with peer review of rough drafts, but I decided to capitalize on it; since I was investing so much extra time and energy into reading and providing feedback on rough drafts, I completely handed summative assessments over to the students. I did perform a quick read-through of final posts in order to see how much effort the student had put into revising and editing the original draft and I did read through the summative feedback to ensure that student evaluations were accurate. Again, this system was very effective, as it removed the burden of summative assessment off of my shoulders (so that I could focus on helping with the writing process), it gave students multiple assessments of their final product, students were much more honest with each other about weaknesses in their writing, and they genuinely valued their peers’ evaluations and integrated them into their revisions and edits for their portfolio.

Next semester, I would like to add a few more layers of feedback to each piece of writing. I am considering, for example, adding a peer review session back into the process, after my own review of their drafts, to encourage a multiple-draft process. I am hoping that I can model effective feedback methods and encourage students to apply them to their own reviews. I would also like to focus more on the finished product and to integrate a postmortem of that product. I already have students write a reflection on each piece that takes their peers’ summative evaluations into consideration and establishes goals for the next piece of writing. But I would like to encourage students to autopsy their products in a much more explicit way after their emotional attachment to the piece has cooled a bit. While some may view the idea of an autopsy negatively, I see value in the process for FYC students.

Let us consider what, exactly, an autopsy is. It is, foremost, a thorough examination that seeks to determine the cause of death. But an autopsy often reveals much more than the cause of death, including diseases or injuries, both past and recent, that are not directly related to the death but that tell us more about the subject’s life and their relative health. But pathologists are not the only ones who perform autopsies. Game developers also perform postmortems. These postmortems seek to identify strengths and weaknesses in the game and to brainstorm how to improve it in future iterations. I think that having students work in groups to perform postmortems on the pieces they select for their portfolios would be an ideal way to encourage them to both value their final products and consider what they could do to add more life to each to make it an even stronger piece of writing. More than a pathologist, I’d like to encourage my students to become like Victor Frankenstein, seeking to collect the best pieces they can find in order to create something greater than the sum of its parts and imbuing it with life. But, rather than being afraid of and rejecting their creature like Frankenstein, I want them to thoroughly examine it and come to love it, both for its strengths and its faults.

 

Embracing the Messiness: Lessons from a 21st Century Classroom

This past Friday, I had the pleasure of presenting at a workshop for regional 7-12th grade teachers. The workshop was sponsored by CoRE, which stands for Collaborative Regional Education, a program my university is developing that will create a partnership between it and regional P-12 schools, other universities, and national organizations and businesses (including Apple) with the goal of improving students’ college- and work-readiness. I was asked to share my experiences with integrating Challenge-Based Learning into my classes.

Because my audience was teachers from all disciplines, all secondary grades, and school systems that run the socioeconomic gamut, I chose to focus on some of the core (pardon the pun) lessons I learned from my experiences, rather than trying to preach or push any one particular method or technology. You can view the presentation slideshow with my notes at HaikuDeck.

 

It doesn’t do too much good to learn something if we then don’t apply it. Here’s a few ways I’m integrating the lessons I highlighted in my talk into my classes this semester:

Trust your students

This semester, my FYC I students have taken over the responsibility of providing both formative feedback and summative assessments for each others’ work. I’m also allowing them free reign when it comes to their blogs, both in terms of subject matter and genres/modes.

My FYC II students are currently busy roleplaying in Second Life (sometimes with me there, sometimes not) and writing the course’s secondary textbook–a guide to roleplaying the roles they are taking on.

My Survey of English Literature students are responsible for teaching each other (and me) about the texts and authors we’re studying this term. They’re also collaboratively writing the final exam.

I’ve pretty much made all of my classes student-centered and given them the responsibility to both guide the entire class’s learning and their own.

De-stigmatize failure

This term, all of my classes are using contract grading. The criteria for each potential grade are directly tied to how much the student wishes to participate and how hard they are willing to work. Want to go full tilt and then some? Contract for an A. Determined to do everything I ask? Contract for a B. Want to pick and choose between learning opportunities? Contract for a C. Both of my composition classes and my speech and debate classes are all using portfolios to demonstrate their work, rather than letter grades on individual performances. The only failure students experience is their failure to live up to the responsibilities and goals they decide to take on.

Peer models

I’m putting extra emphasis on having students identify peers whom they can use as models and indicate  exemplary work using social media (by giving the work a +1, liking it on Facebook, or sharing it with others via Twitter or other sm) and, more explicitly, through nominating them for an A in the course.

Students as co-teachers

As I mentioned, my English literature students are serving as experts on the texts and authors we’re studying this semester. The history major is doing an excellent job of filling us in on the political, cultural, and socioeconomic events that took place and how they might bear on what we’re reading. The women’s studies student is giving us insight into women’s issues of the times and how various texts were responding to them. Others have shared connections between our readings and current texts (such as music by Sublime and Regina Spektor) and issues (such as women in the military).

And it seems like every day a student or two will school me on technology or a new interpretation of a short story I’ve read a hundred times or what the world is like for them and how different their lives and college experiences are from my own. But rather than making me feel even more ignorant of or alienated from them, it brings me closer to understanding and sympathizing with them. And makes it easier to communicate with and guide them. And teach them.

Goodbye, Hello: In Which I Look Backwards Before Going Forwards

photo credit: Avard Woolaver via photopin cc
photo credit: Avard Woolaver via photopin cc

The Fall semester has come to an end and the Spring term is about to begin. Each new term brings with it heightened anticipation as we feverishly map journeys of discovery for our students and blueprint what we hope will be engaging and challenging learning environments. It is a strange season of flux as we look forward with one eye and backward with the other, reflecting on what worked and what failed before so that we know what to recycle, repurpose, and reconsider and what to chalk up to experience. We share much with gardeners, who spend the fallow season plotting and planning, first allowing space for the necessary and the reliable, then squeezing in some untried novelties, deciding what needs to be rotated to revitalize the soil, prepping the ground, sowing the seeds, then waiting patiently for the fruits to flower, tending, weeding, brooding, second-guessing, nurturing, assessing.

Before finalizing my Spring classes, I wanted to reflect, in writing, on some of my more experimental practices from the Fall, especially those about which I promised to post follow-ups.

In “Flips, Cartwheels, and 360’s? Oh my?” I posed the question: “What if I asked my hybrid FYC students to help design a 21st century university?” I wondered if they would be willing or able to accept my challenge. I’m happy to report that they accepted it wholeheartedly and did not disappoint me or the 21st Century Classroom Initiative Committee members who attended their presentations (more on those in a bit). I handed the class a real and intensely relevant problem to solve with no conditions or requirements attached (other than the fact that they had to be able to explain their work in 15 minutes or less). Some of the solutions that students developed were phenomenally outstanding. You can see a sampling of what they came up with at Storify.

In a subsequent post, “This Is What a Final Exam Should Look Like,” I shared my discovery of the research slam–part poster session, part poetry slam–and pondered the questions: “What if final exams looked more like [research slams]? What if students shared their learning with one another in the kind of interactive, experiential, small-group method encouraged by the research slam? . . . How powerful would that be?” Pretty powerful, I thought. And it was. Students arrived early and set up their presentations: a collage of tri-folds, laptops, brochures, and scale models. Small groups of students moved from display to display, as the presenters gave a 15 minutes or less overview of their project and answered questions from the audience. Members of the 21st Century Classroom Initiative were also in attendance, asking questions, jotting down student email addresses, asking for links to presentation materials. I wandered from station to station, filming snippets of presentations and conversations. The room was saturated with voices–discussing, questioning, responding, laughing, debating, critiquing. After such a heady experience, I don’t know that I could ever go back to the traditional final exam–those bent heads; those cramped fingers; those flat, stale pieces of paper; that deathly silence.

In “I’m Bringing Paper Back (‘Cause It’s Still Sexy),” I discussed my plans to strike a balance between the digital and the physical in my classes. I had students digitally and collaboratively annotate one of the texts we read, but I provided hardcopies of their annotations in class and had students use them to develop discussion questions. We also practiced blogging on paper first and students responded so favorably that I plan to have next semester’s classes perform peer review on paper versions of every blog post. I’m slowly falling back in love with paper, especially after reading Jody Shipka’s Toward a Composition Made Whole (which I’ve blogged about a lot recently), and I think it will be making an even bigger comeback next term.

In “Hacking Assessment: Redesigning the Numbers Game,” I continued reflecting on my ongoing battle with assessment. I considered two kinds of assessment, in particular, this past term: peer assessment and contract grading. As I reported in a subsequent post, I ended up giving peer assessment a try in my Basic English Skills class with great success, so much so that it is the primary form of formative assessment in both of my FYC courses next term. Contract grading was less of a success, though that had more to do with my lack of clear communication than anything else. Despite providing exhaustive guidelines, on the end-of-term course assessments several students expressed discomfort with not knowing whether or not each criteria was being met as the semester progressed. On the plus side, I’ve only had two grade complaints so far. I plan to improve my communication with students regarding their progress on grade-level criteria and will provide them with assignment checklists so they can have a visual representation of what they have and have not completed.

In “Remediating Remedial Composition,” I expressed trepidation with some of the radical ideas I had for my Basic English Skills class. Overall, I think the class was a success. Quite a few students disappeared (as is unfortunately typical of remedial classes), but only 4 of the 18 students who finished the class did not receive credit for it. I had to drop the VoiceThread assignment (it was technically too overwhelming in an already tech-heavy class), but the blogs turned out to be very interesting (though not mechanically superior) and I discovered another awesomely invigorating collaborative writing method in the silent dialogues I had students complete in Google Docs (another novelty that will be added to my tried-and-true writing practices).

Overall, I would rate the Fall 2012 semester a success for me, but more so for my students. There were those stellar presentations in my FYC classes giving voice to college students facing a radically revolutionized socioeconomic future and needing a radically revolutionized learning environment to prepare them for it. My Basic English Skills students made great strides in pushing themselves beyond their comfort zones and relying on one another for writing support and nurturance. And my Oral Communication students went above and beyond my expectations as they created public service campaigns that not only raised awareness of important issues but provided a means to act on those issues in positive and impactful ways. I think I’m a little closer to a system of assessment that I believe to be both meaningful and fair. I’ve discovered some awesome techniques to integrate into my composition classes and am especially excited by those that foster collaborative writing practices. And from now on I’ll actually look forward to my final exams rather than dreading and rueing them.

And so it’s time to begin a new semester and a new adventure with a whole new set of experiments and discoveries to anticipate.

“Hoe while it is spring, and enjoy the best anticipations.” ~Charles Dudley Warner

Hacking Assessment: Redesigning the Numbers Game

photo credit: davidfg via photo pin cc

In a recent post, I outlined some ideas that I have about integrating principles of game design into the FYC course. As I pointed out, I’m not all-out gung-ho about the idea of the gamification of education. It turns out that many of my reservations about this latest trend in reforming education are shared by game designers themselves. In her post “Everything Is Game Design,” game designer Elizabeth Sampat makes clear that the assumption that any group of practitioners can co-opt and apply the extremely complex and abstract principles at play in a successfully engaging (to some) game to any other domain is over-reaching:

Gamification” assumes all games share the same mechanics, which means everything that’s gamified is basically the same shitty game. Using badges and leaderboards and offering toothless points for clearly-commercial activities isn’t a magic formula that will engage anyone at any time. Demographics are different, behavior is different . . .

These are the same issues with gamifying the classroom that keep me from wholly embracing the concept. For one, the whole point of a game is that it is . . . well, a game. Games are voluntary. As soon as you force someone to join in a game, it stops being a game for them. It becomes a compulsory activity devoid of intrinsic value and all of the extrinsic rewards you can throw at them, while perhaps artificially increasing their motivation to play the game, cannot turn it back into a game, unless it’s in the negative sense. Even when we gamify a class, we’re still making the learning that takes place within that game compulsory and effectively negating any positive characteristics of gaming that we are attempting to channel. And, as Sampat points out, the characteristics that make any game engaging cannot be standardized. What works for one gamer doesn’t work for another. So, in many ways, game designers face the same kinds of issues and challenges that educators face.

Another point that I think has been largely overlooked in this debate is that, for the large majority of students (if not all), school is already a game. We have goals (behavioral or learning objectives), challenges (in-class activities, homework, exams, and standardized tests), and rewards (grades). We’ve got levels (grade levels based on age in K12 and hours-earned status in college) and leaderboards (A/B honor roll in K12 and President’s and Dean’s lists in college). And we have clearly defined roles (teacher as locus of power and expertise, student as powerless and largely silent novitiate). Some students figure out pretty early how to play the game. In college, these are the students whose identity is inextricably intertwined with their grades. “But I’m an ‘A student,'” they insist when faced with anything other than. Other students learn early on how to game the game. These are the students who know how to manipulate the system and those in charge of it and can often be just as successful at winning the game as their overachieving counterparts. But some students never learn how to play the game according to our rules. Others don’t want to play it because they see it for what it is.

Whether we realize it or not, we’re already playing games with our students. And it’s a numbers game. Play the game according to our rules and we’ll reward you with a high GPA and a diploma, with the promise that these things are the badges you need in order to level up to the American Dream. This kind of game is both irrelevant and counterproductive in a culture that is becoming increasingly participatory, rather than competitive, in nature (just read Share or Die: Voices of the Get Lost Generation in the Age of Crisis to get an idea of how important cooperation and collaboration is becoming for those graduating into the current economy). While many educators are fighting to reform the standardized, hierarchical forms of assessment that have been in place since the industrialization of education, until they are successful at effecting a wholesale paradigm shift and not just applying a false facade and calling it reform, we are forced (much like our students) to try to figure out ways to hack the game. As Sampet argues:

Finding the reward structures and the rules that are already in place, and figuring out how to make them more effective, is the key to making life better for everyone— not adding an additional layer of uninspiring mechanics that push us to engage with mechanics that already suck.

Just as games are not one-size-fits-all, assessment shouldn’t be one-size-fits-all, neither in terms of standardized criteria applied to all students nor evaluative formats used for all courses/disciplines. Just as each course has its own unique set of learning objectives, each course should have a different method for assessing how students go about achieving those objectives. I think it important to explore various assessment methods in an effort to find which is the most effective for a particular course. For example, I have found that a portfolio method is exceptionally well-suited for my composition courses, as it allows for the abstract nature of the writing process and the subjectiveness that characterizes the act of evaluating and valuing a piece of writing. But in trying to incorporate a portfolio system into my speech courses (both an introductory oral communication class and an advanced argumentation and debate class), I have had less success, though for different reasons (perhaps due to the differences among the students: freshman and upper level secondary-education majors, respectively). As much as the portfolio method places value on each student’s individual learning needs, goals, and achievements, within the current grades-based system, students in certain courses need to be able to visualize their learning at both a qualitative and quantitative level. So, what are the alternatives?

Peer Assessment
One option that is gaining ground is peer assessment. Cathy Davidson has successfully explored this method in her “This Is Your Brain on the Internet” class (read “How to Crowdsource Grading” for her description of the process and the thought-provoking debate that followed and “How to Crowdsource Grading: A Report Card” for an overview of her students’ responses to the method). Many MOOCs utilize peer assessment out of necessity. According to Debbie Morrison, within the MOOC environment, peer assessment results in an enhanced learning experience for the student, as grading their peers’ work requires a deeper engagement with course content.

I’ve utilized peer assessment in both of my speech classes to varying degrees and with varying levels of success. In my introductory speech class, the students work together at the beginning of the term to develop a checklist for an effective speech (I don’t use rubrics because, in my experience, they become just another hierarchical form of grading that allows students to retain many of the gaming habits they adopted in K12). They do this by watching several speeches on YouTube and creating individual lists of do’s and don’ts, which we then collate into a master list. For each speech, students are evaluated by five randomly selected anonymous peers, who use the checklist to assess the speech. The students are also filmed and they must use both the video and their peers’ checklists to compose an assessment of their speech that they post to an e-portfolio, along with all artifacts associated with the speech (outlines, bibliographies, slideshows, photos of visual aids, the video of the speech, etc.). For this particular class, I have found that a combination of self and peer assessment has been much more effective than a solely self-based assessment (which tended to be superficial) or even an instructor-based assessment (in which students received only one assessment, as opposed to five, and tended to focus more on improving their “grade” than becoming a more effective speaker). With the peer assessment method, students’ speeches are being evaluated by their audience and their focus becomes oriented towards improving their audience’s response to subsequent speeches.

I have tried this kind of peer assessment in my debate class with far less success. For one, the class is much smaller, and consists, for the most part, of a cohort of sophomore and junior-level secondary education majors. These students tend to be very cliquish and ironically conservative in terms of the practices they expect in the class; they tend to be “A-gamers” obsessed with acing the course and uncomfortable with the level of abstractness and improvisation involved in debate. As a result, they tend to assess their peers over-generously and resist critiquing one another (one class even admitted to giving each other positive assessments across the board because they didn’t want to “hurt someone’s grade”). They look to me as the expert, so their portfolio reflections tend to be focused on flattering me and the course and highlighting aspects of their performances from my point of view (“If I were the instructor, I would give this speech a [insert grade here]”). Despite my best efforts, these students are resistant to assessment formats that are not instructor-based. So what’s a disruptive pedagogue to do?

Contract Grading
While I was at first dismissive of contract grading based on the distaste I harbor for the artificially hierarchical nature of any type of grades-based assessment (and the name’s implications of a kind of capitalistic supply and demand relationship between student and teacher), I have become less dismissive of the method in terms of its ability to bridge the gap between my students’ need for a quantitative value to be placed on their learning and my own objective of encouraging them to recognize and become complicit in the qualitative value of that learning.

For one, I’m hoping that it will eliminate the specter of grades that haunts the course by directly addressing the students’ anxiety regarding their status in a course that has no exams or other easily quantifiable activities. Students will decide what grade they wish to work towards and will have a specific, objective set of criteria that they must achieve in order to earn that grade (yes, I know this sounds just like a syllabus with a traditional grading schema, but contract grading makes the implicit aspects of the traditional schema explicit and, in many ways, mimics the game design principle of starting at zero and gaining points as you go). Once the question of grades is out of the way, perhaps the students will be more willing to focus on learning and improving.

Secondly, contract grading requires student input in regards to the challenges that must be met in order to level-up (yes, I know I’m wading back into gaming territory, but, as I’ve argued, our goal should be figuring out what works for a particular course and cohort of students rather than a wholesale dismissal or acceptance of any one method or theory). Often, in order to earn an A or a B, students must complete additional learning tasks, sometimes choosing between several options, which they can be invited to develop. This aspect of contract grading is the one that I find most promising in terms of encouraging student investment in the learning environment. While I have long preached to students that, in the words of Lennon and McCartney, “in the end, the love you take is equal to the love you make,” contract grading makes student-centered initiative an explicitly integral component of the course.

Thirdly, contract grading will allow me to both address the students’ insistence that I fulfill the role of expert assessor and my wish for them to fulfill the role of deliberate and reflective practitioner. Different grades require different levels of mastery, so students who contract for a certain grade must revise and/or re-attempt assignments that don’t demonstrate mastery. While my debate students can’t re-do a live debate, they can complete a video re-enactment that improves upon their live performance or record a play-by-play self-critique using Voice Thread or screencasting software. In addition, some of the optional assignments can require peer or self-assessment or other types of reflective learning practices.

While I’m not completely comfortable with contract grading (just as I am not completely comfortable with gamification), I also recognize that other assessment methods are not working for my upperclassman and, as a result, are interfering with my efforts to push them beyond a superficial engagement with their learning in the course. I believe firmly that we must recognize our students’ needs, values, and histories; but we can’t pick and choose which of those we take into consideration when designing their learning environments. Sampet makes a point that I think is important for us to keep in mind in the process:

The core principle to remember is that game design is everywhere. Instead of trying to stick a crappy, half-formed game onto real life, the real challenge— the one that’s tough, the one that will bring the greatest results— is to fix the bad game design that’s all around us.

Students won’t be open to assessment that values quality over quantity or process over product until we recognize that our current assessment paradigm is a badly designed game that needs to be torn down and redesigned. Sampet suggests two questions to ask when considering whether or not something is badly designed:

  • What’s supposed to be the goal here?
  • Is this experience set up to help or hinder my ability to reach that goal?
I’m game.
Resources on Contract Grading
These are the sources that I consulted to help me to better understand the possibilities afforded by contract grading: