Using wiki writing for assessment in an Information Technology subject:

Case study B

In this assignment, the whole class used a wiki to write and develop a user manual for a new computer programming language. This task was worth 10% of students' overall marks for this subject.

Web 2.0 tool used: The Moodle wiki tool was used for this assignment.

About the subject

This was a compulsory third year subject in the Bachelor of Software Engineering called "Formal Specification for Software Engineering". The subject provided the formal foundations for the discipline of software engineering. This subject has been offered for a number of years.

17 students were enrolled in the subject in this semester. Approximately 20% were international students.

The subject was taught by one lecturer who had over 35 years of university teaching experience. The development and use of educational technology was this lecturer's main area of research.

Assignment tasks and timing

The assignment was introduced to students at the start of semester and continued for the duration of the semester. The class worked collaboratively to develop a user manual for Event-B, a formal methods modelling language implemented on the Rodin software platform. Students were expected to create the structure of the user manual, provide content and, where appropriate, modify content contributed by others.

Intended learning outcomes

The main learning outcome intended was knowledge of Event-B. This software is fairly state-of-the-art, and like much of this type of software, it is poorly documented. In addition, few learning materials exist to support it. The material upon which the software is based is unfamiliar to students and the learning curve is typically very steep. The lack of resources has hindered learning in the past. Creating the user manual was therefore also seen as a way to improve resources for future students in this subject.

The task was designed to increase students’ reflective learning processes and their level of metacognition. The lecturer anticipated that students would reflect upon their learning experiences whilst they endeavoured to understand the software. Additionally, because the task involved developing a user manual, the students would need to think about the learning outcomes for other students. To do this they needed to be able to put themselves in the position of another student, understand their perspective and then provide an explanation that the other student could learn from, i.e. “what do I know now that I didn’t before and how can I explain that to someone else?” An accompanying discussion board exercise was used to reinforce this reflective process.

This task was different to the typical assessment tasks in this subject, and computing subjects in general, in that students were not required to have the whole User Manual completed. Instead, students were expected to make some contribution to this resource. Rather than a focus on marks for a final product, in this task the emphasis was on value adding and process.

The nature of this task meant that it would be possible for students of all levels to contribute to the wiki and gain marks. Thus all students could go through similar learning experiences. For example, a weak student could add an explanation to help another weak student. They would be in a good position to understand the other student’s perspective and perhaps be able to think of examples appropriate to their level.

A perceived advantage of this task over other more traditional assessment tasks is that it would make students think outside the box and allow them to set their own challenges.

Why Web 2.0?

A wiki was chosen for this task because it allowed for co-creation and collaborative work and was an open publishing technology accessible from anywhere and on any machine.

The open publishing and interactive nature of wikis meant students could contribute and respond to others’ contributions. They were expected to gain insights from reading others’ postings which could lead to further contributions. Students were free to make their own contributions in their own way; however, the need to integrate with what others have written was expected to encourage collaboration.

A key feature of wikis is the emphasis on group ownership over individual activity, which was appropriate for this task. The collaborative nature of wikis was a key to getting a valid and sensible outcome. Before posting the students needed to think, “Does this posting add to the coherence – or detract from it?” Students were instructed that there was an expectation that each contribution would add value in some way. For example, if they deleted content then it should be annotated with a comment such as, “Is this the right way to do this?”

Setting up the assignment

The lecturer set up a web page with a few links to resources. He created a wiki in Moodle and populated it with the first few links, as a paradigm for further student input.

Introducing the assignment to students

The lecturer introduced the task in the first lecture. He explained the timing and weighting of the assessment and the criteria that would be used in assessing the task. The students were told, “If your posting demonstrates that you have learnt something, then you will get marks”. The students were told that they would get one mark for each such contribution up to a maximum of 10 marks. The lecturer was looking for indications of self-learning (what had they learnt on their own) and peer learning (what they had learnt from others).

The task was also outlined in the subject guide, a document which provides students with information about the subject administration, teaching and assessment schedule.

The lecturer explained to the students that the wiki task was a pilot and they were guinea pigs in this experiment. By being open about this he hoped to gain and then keep the students’ trust.

The students were told that copies of the wiki would be kept and used in later years. They were also told that the best contributions would be made available – but would be anonymous. The students would not have access to the wiki history. The students were cautioned that their work would not be changed by the lecturer unless there were blatant errors. The lecturer felt this was important: “Students need to know they can’t use what is on the wiki as gospel.”

The lecturer also explained to the students that any changes made to another student's work should only be done in consultation with the author, and any manipulation of the wiki that might be interpreted as intent to obtain a better mark would be prejudicial to the mark of the student engaging in such practice.

The issue of plagiarism was not mentioned to the students. The students discussed this of their own accord in the discussion board. Given the nature of the wiki, plagiarism was not expected to be a problem. (However, it should be noted that in another similar wiki task in another subject, this time with around 90 students, plagiarism did turn out to be an issue.)

Supporting students through the assignment

The students were provided with links to other web resources to help them with the task. As this was the first running of this type of task, there were no previous exemplars to show the students.

To prepare students for the wiki task, they were required to contribute to a discussion board. Early in the semester the lecturer posted a question to prime the students: “What are the good and bad points of wikis and discussion boards?” This formative component helped students understand how they would be finally assessed and what would gain marks. They could see the summative assessment reflecting the formative assessment.

Students were encouraged to put anything up on the wiki and discussion board. They were encouraged to be open and frank, e.g. ask questions about something they did not understand. In the discussion board students would ask questions such as, “I don’t know ‘x’ – does anyone know how to do ’x’?” and “Here are a couple of ideas – what do others’ think?”

The lecturer monitored the wiki regularly after the first month. At this stage he noticed that not all students had contributed. Attempts were made to get more students involved by regularly mentioning it in the lecture and tutorial.

The lecturer posted several “meta-level” comments on the wiki, aimed at giving students guidance and encouragement. He gave verbal feedback to several students. Three or four students asked him in person for feedback. He advised one student who he thought was off the track, but generally this was not a problem.

Marking the assignment

The lecturer marked the wiki task at the end of semester. The aim was to mark objectively. The marking scheme was based mainly on quantity rather than quality and rubrics were not used. One mark was allocated for each posting that gave an indication that the student had learnt something.

To assess whether contributions were a student’s own work the lecturer looked for correlations between the contributions and what was known of the student. He found that postings were consistent with what he expected of the students. No systematic check was done for plagiarism. A plagiarism detector was not used. The contributions that were made seemed consistent with an understanding of academic attribution and citation practices. The lecturer did not find any examples that indicated students did not understand these practices.

The marks were compared with student performance in other assessment activities. These were in generally good agreement, with the top two students in the class scoring a perfect wiki assessment.

Communicating the results

The marks for the task were posted to the Moodle gradebook. In some cases comments were added to explain the mark given. One student complained about his mark but was satisfied with the explanation he was given.

Evaluating and improving the assignment

The lecturer had been confident that the students would be able to “make a good go of the wiki”. Correlations between the numbers of contributions and final marks suggested that those students who were engaged were the better students overall. Students who contributed a lot of postings did well in their final results. For example, a student who posted 76 contributions got 99%, no student who posted zero contributions got more that 61%, while a student who posted only one entry got 50%.

A strong motivation for the wiki task was the need to build up documentation for the Event-B software. The lecturer noted that this was a very successful outcome of the task. Students who engaged with the wiki went searching for and found information that previous cohorts complained they could not find. The process of building the wiki appeared to be cooperative and collaborative. There were no complaints from students about content being changed or deleted and the lecturer was not aware of any situation where this happened. This was a concern prior to implementing the assignment although the wiki history facility would allow deleted contributions to be reinstated. Students seemed reluctant, perhaps too much so, to change other students' content, apart from typos.

Reflecting on how the assignment had progressed, the lecturer noted that students, on the whole, seemed to respect the added responsibility they were given. No leader emerged in this process; however, there were vastly different rates of activity. Some students made no contributions at all; in other cases, once they started contributing, they made lots of contributions. On the whole the lecturer believed that the wiki task was more effective for students’ learning than traditional assignments. He felt that students were thinking at a level which was not directly concerned with the content. They were operating at a meta-level: using the content as a vehicle for expressing their meta-level understandings.

The lecturer found the task provided convincing evidence that if you can get students to accept responsibility for undertaking their own learning then it is easier to satisfy the learning outcomes. He stated, “I have never seen an example of a student that had attempted an (open-ended) challenge who had walked away learning nothing. On the other hand I have seen plenty of students who had attempted a carefully laid out assignment and then ended up making an absolute pig’s breakfast of it.”

He also noted:

“A big challenge in teaching is to get the students to understand that we want them to think creatively. A problem with students is that they want to know chapter and verse of what they need to do. They do not expect to have to think creatively about a subject. For example, a question asked by one student, ‘Are there any resources on line where you specify exactly what you expect of us?’”

The lecturer used an open approach to implementing this task as it was the first time he had set a task of this nature and he was unsure of how the students would respond. He considered it was a learning exercise for him as well as the students. He was able to learn from this experience. In the following semester he taught a second-year subject with a larger enrolment (90 students). This was an established, stable subject with many available resources. He decided that with this group he would use a more structured approach. He decided to encourage students to focus more on content construction rather than just cutting and pasting information, which is what had happened with the third-year subject. He also decided that he would monitor these students more carefully during the semester. With the small third-year group it seemed reasonable to just assess the end product. With the second-year group, a more mercenary attitude was evident. Students wanted to know what marks they were getting.

The lecturer felt that a big advantage of the wiki task was that it gave students experiences and opportunities for their learning that are not available in traditional assessments. He believed the open nature of the task encouraged academic honesty: “The students do not want to appear a plagiarist to their classmates.”

Selected documents from this case study

An extract from the final version of the wiki: Example of wiki content.tiff

Page source:
Gray, K., Waycott, J., Thompson, C., Clerehan, R., Sheard, J., Hamilton, M., & Richardson, J. (2011) Using Social Web (Web 2.0) Activities for Student Assessment: Resources for University Learning and Teaching. Retrieved from https://web2assessmentresources.wikispaces.com