All Together Now, Part 3: Crowdsourced Assessment Using Google Forms

By matt_leclair

In past posts, I’ve written about various ways that I use Google Docs (now Google Drive) in my courses, including collaborative writing and crowdsourcing annotations for the texts we are reading. Recently, I’ve experimented with a third use for another tool in the Google Docs collection: using Google Forms to crowdsource assessments of the students’ blog posts.

I’ve regularly discussed my struggles with assessment. This semester, this struggle has intensified as I have found it increasingly difficult to manage  assessing and providing feedback on students’ work. This has some to do with the fact that I am teaching five classes, three of which are composition classes. But it also has a lot to do with the fact that all of my classes are now using the challenge-based learning model, so the work that students are doing is both more challenging and complex. This is especially true now that we are near the end of the term because this is where the most creative and cognitively dissonant work is done. I have found it difficult to adequately divide my attention between their regular writing assignments and the work they are doing behind-the-scenes. In trying to figure out how to take some of the onus off of myself without sacrificing timely feedback, I immediately thought of Cathy Davidson’s method of crowdsourcing grading. But as I’ve mentioned in my previous posts on assessment, I’ve met with some resistance from students who don’t want the burden or responsibility of providing negative assessments of their peers:

As a result, they tend to assess their peers over-generously and resist critiquing one another (one class even admitted to giving each other positive assessments across the board because they didn’t want to “hurt someone’s grade”).

One method that I have found to be relatively successful for overcoming these feelings is by making all assessments anonymous, especially in low-stakes, informal situations such as peer review. In considering how I could formalize anonymous peer assessment, I immediately thought of Google Forms. This Google app allows you to create a form that includes various types of questions, such as multiple choice, checklists, and open-ended. Once the form is completed by a respondent, the answers are automatically transferred to a spreadsheet. The creator of the form can then manipulate and share the results however they wish, including an option to view the results as a graphic summary.  The sharing options are useful for sharing the assessment results with students and the summary option is a quick way to get an idea of overarching issues within the students’ work (as well as what the students’ strengths are).

Since it’s so late in the term, I decided to pilot peer assessment rather than integrate it as a formal course assignment (students are required to complete at least 2 assessments, but I am not assigning which peers they must assess). I created a form that is based on the list of criteria for a good blog post that the class worked together to create at the beginning of the term. In addition to these items, I added two open-ended questions that require students to offer some anecdotal feedback on their peers’ posts. Here is the form I created and an excerpt from the results summary:

 

 

 

Once I received the results, it was easy to share them with the students. I simply filtered the column for the title of post alphabetically so that all entries for a particular post were together. I then hid the column for the assessor’s name. Next, I selected the cells that applied to a specific post and downloaded the selection as a PDF that I emailed to the student (I wanted to include the column titles in the selection, so I simply moved down the spreadsheet, hiding the rows for each student’s post after downloading them and before selecting the next set of cells for the next post).

What I have found so far is that using Google Forms is an effective method for crowdsourcing assessment of students’ writing. Firstly, it’s quick and convenient for students to complete the assessment. Secondly, it allows for anonymity, eliminating students’ fears about offering negative feedback that may hurt their peers’ feelings or impact their interpersonal relationships with them in and out of class. Lastly, it provides authors with multiple pieces of feedback on their writing that is simply organized. The fact that some of these pieces of feedback may focus on different aspects of their work and/or may compete with one another is actually a positive, as it helps authors see how different readers focus on different aspects of a piece of writing and have different expectations and needs. I think this kind of assessment is also especially effective because as I tell students, when they write a blog post, I am not their primary audience; rather, their peers and anyone else who might be interested in their topic are their target audiences. By receiving feedback from their peers/audience, this reality is made tangible to them.

Advertisements

4 thoughts on “All Together Now, Part 3: Crowdsourced Assessment Using Google Forms”

  1. This is very interesting. I’ve been working to incorporate group blog assignments into my comp classes over the past few terms, and I’ve never come up with an assessment process that has “felt right” to me. I’ve been reluctant to practice peer reviewing / assessment because, as you point out, students tend to be overgenerous or else protest that they should not be required to participate in such a process. What — if any — incentive system do you use to ensure students participate in these assessments? I’m always reluctant to allow points just for completing an assessment or peer review, but I’ve often found that’s the best way to ensure high participation.

    1. I actually don’t offer any incentives. I’m using contract grading for this particular class, so completing the assessments is one criteria for an A, B, or C in the course. Either they do the assessments or they don’t meet the criteria for one of those grades. When I use peer review, I don’t count it at all. My thinking is that students who don’t bring a draft to peer review are penalizing themselves and I have found that assigning points to drafts doesn’t necessarily encourage students to bring a genuine working draft (students who are just doing it for the points will just throw something together at the last minute). Students realize fairly early on the benefits of peer review and I would much rather them voluntarily participate and bring in substantial working drafts than just half-ass it for some points. If a large number of students come to the first peer review session without drafts (thus throwing the ratio of drafts to reviewers dramatically off-balance), I simply ask those who did not come to class prepared to leave. This suffices for a wake-up call for the majority of students and I have no problems with this issue after that. I’m not concerned with the small minority of students who don’t bring in drafts. In removing grade-based incentives from my FYC class, I send a clear message about the kind of motivation I expect and those students who won’t do work without those incentives quickly disappear. The rest generally do much better work because they no longer have/need extrinsic motivation, but are writing for themselves rather than a grade or points. The same applies to peer review and assessment–doing the work is about getting better as a writer so if they want to do that, then they’ll participate in those activities in an authentic way.

      1. Very interesting. Our system requires students to bring a complete draft, because the primary grade comes from revision and reflection. We’re not free to implement the sort of grade contract you outline here, and there’s a lot of concern about being able to document, mathematically, the reasons for students succeeding or failing in many of these classes. There’s a sense of growing frustration with this, and I think the program directors are assessing how / when to implement changes to the system. But for now, I’m stuck with finding ways to create incentives for many activities that should be voluntary / etc. In order to make the peer review process a little more substantial, I often assign it as a take-home project — students exchange papers, take them home and review them based on guidelines, then write a brief formal letter that offers analysis, revision suggestions, and a little praise to boost spirits. I don’t grade these, I simply check them off.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s