2.13.+Using+wiki+writing+for+assessment+in+an+Education+subject

=Using wiki writing for assessment in an Education subject: A case study = In this assignment, students - who were academics enrolled in a Graduate Certificate in Tertiary Education program - used a wiki to write five personal accounts of aspects of student assessment, then a three-stage analysis of each of these accounts and finally a reflective commentary. The assignment was about 2000 words and worth 100% of the student’s mark in the subject.

Web 2.0 tool used: The Pbworks wiki tool (unlicensed freeware version) www.pbworks.com was used for this assignment.

About the subject
This subject was a key component of a Graduate Certificate in Tertiary Education foundations for teaching at university program, entitled "Student Assessment". The subject covered assessment design, strategies and practices for assessment and moderation, and institutional assessment processes and practices.

The subject ran during a single semester as a full-day workshop, a half-day workshop one month later and regular online activity. However it could be completed flexibly over a one- or three-year part-time study period. So some students attended no workshops, conferring one-on-one with staff instead. The wiki writing assigment had been offered previously.

There were about 20 students active in the subject and about 20 more on the class list who had deferred completion. All were new teaching staff in a range of departments at this combined higher and vocational education university, i.e. they were colleagues of the staff who taught them. Their continuing employment was conditional upon completing this qualification. They had a wide range of demographic characteristics and employment histories and most had postgraduate qualifications in their discipline.

Two experienced educators team-taught this subject.

Assignment tasks and timing
Each student wrote five personal accounts of aspects of student assessment, centred on their own work practices and supplemented by educational resources and research sources. The student then wrote up a three-stage analysis of each of these accounts, using a structured approach called “sketch, thread and theorise”, i.e. highlighting key words and phrases used in the account, making a list of keywords or key concepts, and commenting on the personal theory that this list suggested. They also wrote a final reflective comment about their learning in the subject. The assignment ran over a three-month period.

Intended learning outcomes
The learning outcomes matched the university’s statement of teacher capabilities. Teaching academics at the university were expected to:


 * //know and respond to the range of learning needs of ... students both in terms of the knowledge they bring and the learning approaches they prefer//
 * //design assessment that identifies and reports student learning, provides feedback to student about their learning, and informs further planning of teaching and learning//
 * //provide prompt, informed and constructive feedback to students on their work//
 * //ensure consistency in assessing student work by applying consistent criteria and standards in making assessment judgments//
 * //monitor student achievement and maintain accurate records of their progress//

Why Web 2.0?
The assignment was based on the idea of “praxis inquiry”, i.e. teachers present evidence from their own practice for their assertions about the best ways to teach, justified by analysis of that practice and by support from the research and policy literature. The wiki format (including highlighting and commenting tools) supported students to manage and set out a staged inquiry.

The wiki was used to create a sense of collaborative endeavour among all staff and students throughout the degree, including those students who could not attend face-to-face sessions. Also it let them draw from their work in this subject later, to relate it to other subjects in the degree. The wiki made it convenient for students to give and receive feedback. This gave them an authentic experience of the scholarship of teaching. “Assessment made public and visible in this way emphasises assessment as learning rather than assessment of learning.”

Setting up the assignment
At the start of the subject staff updated the introductory information and instructions in the subject wiki, and enabled access for enrolled students.

Each student already had a personal wiki site, set up by the course coordinator prior to the workshop. Separate wikis were used for every subject in the degree. All parts of the subject wiki were available for reading and comment by all students throughout the subject. However, only the student “owner” of a wiki, and staff, were enabled to edit their site.

Introducing the assignment to students
At the start of the one-day workshop staff handed out and explained at length a subject guide with assignment details and a feedback sheet with marking criteria. They gave a mini-lecture on the importance of making assessment expectations transparent to students and to other staff. This included philosophical ideas about the difference between public and private spheres and interests as a rationale for the wiki writing assignment.

The lecturers used a web-connected computer with a data projector and wall screen to show and scroll through and discuss student wiki work from previous semesters in this and other subjects. They asked students to use a practice worksheet and coloured pens to start drafting and sharing ideas in small groups, while the lecturers circulated around the room and responded to questions and comments. They explained the structured approach to writing required in the wiki as a way of visualising one’s thinking and showing movement in one’s ideas over time. They also promised further benefits from the wiki: “in the wiki you’ll see we have resources about assessment” and they took photos of small groups in action and let the class know that they would upload these to the wiki later.

For the second half of the day they moved the students from a seminar room to a computer lab. Here they asked students to start writing in the wiki by building on paper-and-pen practice earlier in the day, to “get into the idea”.

Supporting students through the assignment
Immediately after the workshop, staff updated an instruction page on the wiki with questions and answers that had arisen. They also wrote an individual response in every student’s wiki.

Staff scheduled regular times each week when they would make themselves available to students specifically for “wiki support”.

Students were encouraged to comment on other students’ wikis at any time. Students who had not attended the workshop were particularly expected to “have a conversation”, i.e. make a number of comments on other students’ work.

Staff routinely monitored wiki activity and used phone or email to follow up students who were inactive. They added comments from time to time especially in the lead-up to the assignment due date. Throughout this assignment, staff were modelling for students - who were teachers themselves - how staff can interact well with students. They took great care with their written feedback on student work in the wiki (e.g. in spelling, ideas, tone, accuracy) because “it stays there”.

Marking the assignment
Towards the end of semester, the two lecturers met in a three-hour face-to-face meeting to review each student’s wiki and determine whose work was ready to correct.

Staff inserted into these students’ wikis a feedback grid that could be edited by all students. It showed the five learning outcomes with four possible bands of achievement for each. Each student was expected to use this grid over the subsequent fortnight to comment on at least one other student’s wiki and to place this comment in a rating band.

During their meeting staff worked together to give feedback in a few students’ grids and agreed on those to do individually afterwards. Staff read peer comments and exchanged comments themselves in these grids, over the subsequent fortnight. By this stage, one of the lecturers said, “I have in my head where people are at.”

The staff met again to check for evidence to support the allocation of a proportion of the total mark at the predominant band where feedback regarding each criterion had accrued, and then calculate a total mark (five criteria weighted at 20% each) for each student. They started from the middle (e.g., where a student’s work was rated in the HD band 85-100, they would start at 90 and work up and down in terms of considering what the student had achieved). If there were gaps or differences in feedback, they looked beyond the grid, going back to the original work and negotiating where they thought the result should be placed. This stage of marking was not meant to take too much time, recognising “a dimension of subjectivity”: “it was never going to be a precise science”.

Communicating the results
Results were submitted to the university system by the end of the semester for those students who had completed. Staff posted a confirmatory comment in the student’s wiki, e.g. “congratulations on completing the subject”. Students received their exact marks privately by email, or they received an email confirming that their work was still in progress.

If a student were to come back and ask for a higher mark, then a dialogue about whether the student wanted to do more work or not and options for what to do would occur in private, out of the wiki.

The dialogue between two markers in each student’s wiki was evident to all students. The appearance of the markers’ dialogue generated a surge in peer feedback activity.

The evolution of shared reasoning about each student’s final mark was made transparent through the use of the feedback grid in their wiki.

Evaluating and improving the assignment
Students would normally give feedback using the standard anonymous online student feedback form for every subject in this institution, but this was not timed to fit with the way the subject ran. Therefore, an informal feedback form was developed and staff asked students to complete feedback questions by email. Feedback was collated and fed back via the subject wiki and handouts to prospective students.

The subject underwent informal annual review and five-yearly formal review along with other subjects in the degree program.

Staff regularly presented the design of the program at peer reviewed forums in the university and nationally.

The lecturers felt that most students achieved at a high level, at least a “relational understanding” level, although some students didn’t appear to be concerned about getting to the highest (“abstract understanding”) level of achievement.

To keep up with the interaction demands of this assignment staff had to put aside a lot of time. Also they had to judge carefully when they should play a part in the wiki and when they should let students run the discussion. The lecturers found that many students were hesitant to make judgments about peers’ wikis and needed more scaffolding before they would do this.

Previously, when students completed the subject, all had been asked for – and some had given – permission to keep their wiki sites visible to incoming students, and the original subject wiki could be viewed from one semester to another. However staff were unsure whether the net effect of this was to build a learning community or to give an unfair advantage to subsequent students. Henceforth, staff planned to “clone” the subject wiki at the end of each semester. This way, they would archive a complete record of student work but would make a new subject wiki each time the subject ran, with only student work still in progress and selected exemplars visible to new students.

The university’s IT policy was silent on the use of a wiki tool outside the university’s management and governance.

Selected documents from this case study
Extract from the Subject Outline provided to students, detailing the learning objectives and aims of the course: Outline of Subject_revised.pdf

Instructions provided to students on the wiki about "wiki protocol": wiki protocols instructions.pdf

 Page source:  Gray, K., Waycott, J., Thompson, C., Clerehan, R., Sheard, J., Hamilton, M., & Richardson, J. (2011) //Using Social Web (Web 2.0) Activities for Student Assessment: Resources for University Learning and Teaching.// Retrieved from https://web2assessmentresources.wikispaces.com