Using wiki writing for assessment in an Information Technology subject:

Case study A

Students used a wiki to compile two online journals in which they recorded their progress in two computer programming assignments. Each journal was worth 10% of the subject grade and was a component of a separate assignment worth 35% of the final mark for this subject. Students completed these assignments individually.

Web 2.0 tool used: A wiki tool implemented in Moodle was used for these tasks. Moodle is one of the main learning management systems used in the University.

About the subject

The subject, called "Programming for distributed and parallel mobile systems" was a Masters level subject taught in the Master of Information Technology course. The subject focused on the design and programming techniques essential for developing distributed software systems and applications, and was taught using the programming language Java.

There were 41 students enrolled in the subject, predominantly international students.

The subject was taught by one lecturer who had 10 years experience in teaching programming subjects at undergraduate and postgraduate level.

Assignment tasks and timing

Students were given two programming assignments to complete individually during the semester. The first assignment was released in Week 3 of the semester and the second was released in Week 8. The students were given four weeks and two days to complete each assignment.

Both assignments involved various components, including writing an online reflective journal in a wiki. In this journal students were expected to log their activities and reflections as they worked on the assignment. The assessment for each assignment was by interview with the tutor. In these interviews students were required to answer questions about the programs they had submitted. As part of the wiki task, students were instructed to offer suggestions for potential interview questions.

Intended learning outcomes

This assignment had several intended outcomes relating to student learning.

The lecturer believed that requiring students to record their progress would encourage them to start working on their assignment early thus avoiding any last minute panic and helping them to complete their assignment on time. The potential for procrastination would be avoided by getting them to start thinking and, more importantly, documenting points about the task as soon as possible after the specification was released. This would also help diminish initial fears and build confidence in approaching and tackling what may seem, at first, daunting tasks.

The lecturer also anticipated that this assignment would facilitate reflective learning: by recording their experiences students would be encouraged to reflect on their learning. This would help promote conscious thought and effort in breaking down an assessment task into logical steps, helping students identify the sequence and knowledge and skills needed to complete each step. Reflection would provoke a deeper level of engagement with the subject matter, rather than the mechanical process of implementing a program solution. Reflecting on the skills and knowledge needed to complete the tasks and documenting those reflections would help students learn, and be aware of exactly what they have learned from the tasks as they implemented them. Students would gain a sense of satisfaction as they become aware of what they are learning of the topics presented in the course.

The experiences gained from this task would be transferable to future professional programming problems and tasks as well as research.

By giving students the opportunity to create their own interview agenda they would feel a sense of empowerment. This would also help reduce any anxiety associated with the interview assessment.

From another perspective, this assignment was also intended to give the lecturer insight into whether a student’s perception of what they were learning from their implementation of the program was what the task was designed to teach them.

Why Web 2.0?

A wiki was chosen because it was the only tool in the Moodle suite that could be made visible only to the individual student and the lecturer.

A Web 2.0 technology was chosen for this task because the lecturer believed students were familiar with the medium and would find it a novel medium for completing an assignment. The lecturer also believed that students would be uninhibited and unintimidated by the technology and be able to express themselves freely: “Web 2.0 is the technology of the current student cohort”.

In addition, the wiki format was chosen because it allowed students to express their ideas and opinions in a variety of formats; for example using hyperlinks, tables, headings, bulleted lists. Students were given no restrictions or guidelines as to how they should use the wiki; they were encouraged to explore the wiki technology to find a format that suited them.

The wiki was used in a restricted access mode because the assignment required students to record a chronicle of their personal experiences and reflections in implementing their programming tasks and there was no reason for others to see these. The lecturer felt that students would be influenced by the ideas and opinions of others if the wikis had been made publicly visible.

Setting up the assignment

Before the assignment began, the lecturer had to create a wiki resource in Moodle and then set up access to make it visible and editable by each individual student and the lecturer.

Introducing the assignment to students

The wiki task was part of two larger assignments, and was described at length in the assignment specifications. The wiki task was also explained to students in the lecture, for both assignments. A trial wiki was set up to demonstrate the concept of a wiki and show students what was required. Beyond this there was a lot of scope for the students to use the wiki in any way they saw as appropriate.

Supporting students through the assignment

The lecturer had intended to give feedback in each student’s wiki as the assessment tasks progressed. However, when he completed the weekly checking he found that many students had not done anything to report on. In these cases the feedback was in the form of a reminder that the task was assessable. In many cases students did not submit entries in time to allow for feedback that could have been useful to them.

Marking the assignment

The one lecturer who taught this subject also completed all the marking.

The wikis were assessed after the submission of each assignment. The marking criteria considered the number of entries, the distribution of these over the course of the assignment, and the quality of the entries in terms of the extent to which they met the requirements stated in the specification.

Each wiki task was worth 10% of the final marks for the subject. The rubric used was as follows:
- a maximum of ten entries were assessed;
- a maximum of three entries were chosen from each week; and
- ½ mark was allocated for each entry and ½ mark for the quality of the entry.

The grade obtained for the wiki task could not be more than the grade obtained for the programming implementation component. However, the mark for the wiki could be substantially less than the mark for the programming component.

The students were provided with the guidelines and criteria for the different grading levels in the specification for the assignment.

Communicating the results

The mark provided for each wiki task was part of the overall grade students received for each assignment.

After each assignment the lecturer provided feedback to the whole class on the performance of the wiki task. In addition, some students were given specific detailed comments on request or when a specific issue became apparent.

Evaluating and improving the assignment

The lecturer was able to collect evidence about how the assignment worked by reading the wiki postings and assessing the effectiveness of the task in helping the students achieve the learning outcomes. In other words the wiki journals provided evidence about whether students were learning what was intended from each assignment. For the students who used the wiki as intended, it was possible to gauge progress on the assignment and determine whether it was the student's own work or not. One inadvertent outcome of using the wiki was that the lecturer was able to double check student claims about what they had or hadn't done: One student denied having been told of particular assessment criteria relating to program design. However, she had recorded the information about the criteria in her wiki. Once a contribution had been put in the wiki it could not be completely erased; it may be deleted but could be recovered from the archive of the wiki history.

The lecturer did not obtain specific feedback from the students about the wiki task. Upon reflection, the lecturer felt the rationale for the task was sound. The wiki provided a more immediate way of monitoring how students were progressing, thinking, learning, and engaging with the assignment. It enabled individual feedback to students in a way that was fresher, more immediate, and spontaneous – in the spirit of a wiki.

The lecturer suggested that an open rather than a restricted access wiki may have been better because seeing other contributions could have motivated students to contribute, inspired ideas, and encouraged more reflective comments. Students could have also benefited from seeing what other students thought the learning task was about.

There were a few issues with the way students responded to the wiki tasks. The main aims of the wiki tasks were to get the students started on their work early, encourage steady progress throughout the semester and to facilitate reflective learning. However, the lecturer did not feel that these aims were achieved. This may have been due to the characteristics of this cohort of students. The lecturer observed that the students had not been amenable to this task and suggested that it may have been due to their lack of experience in reflecting on their learning and designing interview questions.

A few students did not use the wiki at all. Of the students who used it, the vast majority did not use it in the way in which it was intended. Most students only had a few entries at best and a number produced something in the last few days before the deadline. One student emailed his one and only contribution after the wiki had closed. Considering the weight of the assessment task (20% overall assessment for the subject) it was surprising that most did not take it more seriously.

Students showed little evidence of reflection. Most entries were just statements about what they had done. Furthermore, only a few students contributed questions or tasks for the interview. The lecturer believed that this may have been because students were not used to having the opportunity to be empowered in their learning. He had the impression that the wiki task was an afterthought for many of the students.

While some students put in a reasonable effort, incorporating hyperlinks and images into their wikis, most students did not use a variety of formats that wikis allow.

The students’ performance in the wiki task paralleled other aspects of the assignment. Although confident with the subject content, many students used technologies and instructional design environments that they were familiar with from other subjects rather than learning something new. The students appeared to use a surface level approach to reading the fine grained requirements specified for the assignment and a number of students submitted work that did not satisfy the functional requirements.

Although the task was explained in different ways and students were shown examples of what was expected, not all students seemed to understand what was required. Considering the way the students approached the wiki task a conclusion from this study is that it is important to educate students as to the importance of this type of task and the potential benefits to their learning.

Selected documents from this case study

Information provided to students about the wiki assignment: Wiki--Assignment-1-Specification.pdf

Page source:
Gray, K., Waycott, J., Thompson, C., Clerehan, R., Sheard, J., Hamilton, M., & Richardson, J. (2011) Using Social Web (Web 2.0) Activities for Student Assessment: Resources for University Learning and Teaching. Retrieved from https://web2assessmentresources.wikispaces.com