2.12.+Using+wiki+writing+for+assessment+in+an+Accounting+subject

=Using wiki writing for assessment in an Accounting subject: A case study =

In this assignment, student groups wrote reports on the financial performance of major companies based on publicly available information. This assignment was worth 25% of their overall mark in the subject.

Web 2.0 tool used: A wiki tool developed in-house by the Faculty’s IT staff and integrated into the University’s Learning Management System (LMS).

About the subject
The subject was called "Financial Statement Analysis" and was taught in Accounting and Finance. It could be taken as an elective in any year of a coursework Masters degree. The subject offered:

//An examination of the role of financial statement information and analysis in the evaluation of the current profitability, liquidity and riskiness of public companies, and the prediction of their future condition by parties external to the firm.//

There were 118 students in two classes of about 60 each enrolled in this subject in the first semester of 2010. Almost all the students were on international visas from southeast Asia.

One staff member ran this subject. This staff member had over 10 years’ university teaching experience and about 20 years’ industry experience. His innovation with using a wiki in this subject was the focus of a university-supported learning and teaching project.

Assignment tasks and timing
At the beginning of semester, students had to form groups with 2 to 6 members. Each group was allocated a “Top 50” Australian public company to report on in their wiki. Each group had to create a wiki site for their company and add to it week-by-week based on research into publicly available information about the company such as annual reports and media reports. Information added each week had to be directly related to the financial analysis concept or principle that was the focus of that week’s lecture, e.g. week 2 Analysing company balance sheets; week 4 Analysing company cash flow statements. At the end of the semester, the group had to compile an executive summary of the wiki in the form of a Word document, and submit this for assessment by the lecturer. Students also undertook a peer review assessment of another group's wiki at the end of semester.

Intended learning outcomes
The main intended learning outcomes of this assignment were specialised knowledge or skills required in the discipline or profession. Specifically, students were expected to build skills in writing a comprehensive evaluation of a company’s financial performance based on systematic review and analysis of evidence. There were also ‘process’ aims; that is, students were expected to improve generic skills such as: accessing data and research information from a variety of sources, written communication and collaborative learning.

Why Web 2.0?
A wiki was chosen for this assignment to enable many small groups to build up complex evaluations of one company in small steps over a period of time. It was anticipated that students would be more engaged and interact more with the subject matter than in some other types of assignment, and that they would find the experience of working in groups ‘comforting’. Using a wiki also allowed for groups to name themselves distinctively as part of building their shared identity.

Setting up the assignment
A ‘test wiki’ was set up for the lecturer and technical support staff to use to check how the tool was working.

This assignment was redesigned to use the wiki format for the first time in this semester.

The possibility of "recycling" material compiled by students from past semesters was designed out, by the assignment focus on up-to-date information, i.e. the companies allocated to groups would change from semester to semester, and the information about their financial performance would change from day to day. It was not intended that academic sources or research publications be used in this assignment, but rather the information sources and formats used in routine professional practice.

The lecturer anticipated that there would be some student "free riders learning from others’ work without having shared their own". He therefore chose to make student group wikis visible only to group members and the lecturer for 11 of the 12 weeks in the semester to encourage an original approach to compiling the wiki content and to discourage exploiting other groups’ ideas.

Introducing the assignment to students
Information about the assignment was "drip fed", starting with its description in the subject guide, followed by "talking it up" in early classes, then giving out a two-page handout along with a five-minute overview from the lecturer.

The handout set out five criteria for peer assessment that mapped closely onto the six purposes described in the handout. A structure for the wiki content, closely following the weekly lecture topics with two additional headings, was recommended.

Students were expected to form their own groups and each group was randomly allocated a company to work on.

Supporting students through the assignment
There were no previous semesters’ work to draw on, but the lecturer used several companies not included in the assignment as models, presenting the kind of information about them in class, linked to each week’s lecture, that student groups were expected to present about their allotted company in the wiki.

The lecturer "quietly" monitored the wikis each week and set aside time in weekly lectures for reminders and a question and answer session with students about general progress, including about technical issues that arose.

A few students consulted privately with the instructor about the assignment out of class hours. The lecturer wrote no formative feedback in the wikis because he felt students needed to take responsibility and to understand that he would be ‘not looking over their shoulder.’

These postgraduate students were expected to show more self-reliance and initiative than if they were undergrads, so it was assumed that they would work out for themselves how to use wiki editing tools and features without any specific instruction and to design their own response to the assigned task.

Marking the assignment
At the end of the semester, all wikis were made visible to all students. Students were expected to spend 30-60 minutes doing peer review which involved looking through one or more wikis with upwards of 20 pages of content. Peer review ensured ‘additional pairs of eyes’ on each wiki’s content. Each wiki was reviewed by several students from different groups. Students gave the wikis they reviewed a mark out of 25 based on:

• original and informative summaries of links or pasted information from other sources • regular contributions • risk assessment of the company • evaluation of the performance of the company • accounting issues that required adjustment to the financial reports for more purposeful analysis

This was expected to include random checking of wiki content and links for any issues with originality. Peer reviewers were anonymous and randomly allocated so as to maintain independence and avoid favouritism.

Peer reviewers’ marks were scaled by the lecturer to yield 10% of the total mark. Peer reviewers were "quite harsh" – the lowest result was 6.6 % and the highest was 9.15%. Not all reviewers gave comments and many comments were one-liners. Peer reviewers were not expected to provide extensive feedback: the lecturer thought it might "overburden" reviewers if they were expected to comment in depth.

The lecturer marked wiki executive summaries out of 60 and scaled this to be worth 15% of the total mark. At the end of the semester, each group had to submit their wiki executive summary (27 in all) electronically as a Word document to the lecturer, who printed them off ("much quicker than on-screen and I couldn’t make as many comments on-screen"). Marking took about half an hour per assignment. The lecturer read them one at a time with a printed scoring sheet beside him, which allowed up to five marks for each of ten substantive criteria (such as an analysis of the cash flow statement, e.g. operating, investing and financing activities) plus up to ten marks for report-writing elements (e.g. organisation, references, use of tables, appendices). He also annotated each paragraph of each summary with emphases and comments – for instance “what does this mean?” or “what conclusion can you draw?” and a minor level of checking of calculations, e.g. in tables of company accounts.

The criteria used for lecturer’s marking of each group’s executive summary were deliberately withheld from students until late in the semester as part of a strategy to discriminate between average and above-average students. The lowest mark given out was 44/60 and the highest was 57-58/60. Where it was obvious that groups had given certain parts of the work to certain members to do individually, delegating bits with no joint review of the finished report, lower marks were given - one group was very good but others were fairly poor in this respect. The same mark was given to all students in a group that produced a wiki.

After marking the first few summaries, the lecturer reviewed marks allocated so far, and "got into the zone of marking". Thereafter he did not look back over any more of the marks he assigned. He also factored peer review results into his calculation of the group mark.

The lecturer made the final exam, worth 50% of subject mark, harder than in previous semesters, in order to get a clear spread of marks. (The marking allocation was 20% for mid-term test; 50% for final exam; 5% for classroom participation; and 25% for the combined wiki assignment – 10% peer assessment and 15% lecturer mark for executive summary) The School where this subject was taught used a normal distribution of marks so that any results off the curve would have to be defended in examiners’ meetings.

Communicating the results
Individual students could find their own mark for their participation and wiki assignment in the LMS gradebook. After the marking period, students could pick up one photocopy of the marked group summary from the instructor’s office. However many students had gone home overseas by then.

Evaluating and improving the assignment
To evaluate the assignment, student feedback on the following questions was specifically sought:


 * 1) Was the wiki assignment a useful means of adding to your learning experience of the subject content on a week by week basis? Was it too long?
 * 2) Did you find that the task of assessment of other groups’ work was a useful learning exercise for yourself and added to the depth of your understanding of the principles of the course? Yes / No – any comment?
 * 3) Did you find the five items / criteria for assessment of another groups’ wiki assignment a useful means of assisting your task? Yes / No – any comment?
 * 4) Did you find that the group exercise enabled all members of the group to participate and make relatively equal contributions? Or did you find that only one or two members did most of the work? Any comment? Were the groups too big? The right size?

Consistent international student demand for the subject in future was expected to be an indicator that the assignment was well regarded.

The lecturer planned to monitor and compare student results from different patterns of group contribution against their individual results in the subject final exam, to determine what was academically effective that could translate into study skills advice to students.

The lecturer also planned to present details of this assignment to a staff forum for collegial feedback.

Reflecting on how well the assignment worked, the lecturer noted that the challenge of "the unknown" – what kinds of news items and other relevant information sources each group would find to link and analyse in their wiki – was met and exceeded in the work that was produced.

Contrary to expectations, some students found it hard to set up and manage the content of their site using the wiki tool.

Approximately 13 students out of 118 surveyed gave feedback, mostly very positive, with some making constructive suggestions such as: wanting more definitive guidelines for presentation of the wiki site; wanting a few milestones earlier in the semester to review and share wiki progress; and four or five students being the optimal group size.

The lecturer felt there was a need to give more emphasis to academic integrity, specifically to give students instructions about: • interpreting what they had copied and pasted • making more distinction between their own and others’ content by using a different font or style to set them apart • limiting the total amount of content that could be copied and pasted

The peer review task needed to be brought forward because it was too close to the end of semester exam period to allow time for staff-student and student-student interaction and subsequent student learning.

The lecturer had hoped that LMS statistics would give good insights into group and individual activity over time. However students’ way of doing work off-campus and then uploading it in a batch from an on-campus computer meant that the wiki activity log didn’t give a good picture of the actual times when students were working on the assignment or the actual effort each individual had put in.

Selected documents from this case study
The handout provided to students to introduce the wiki assignment: WikiAssignmentHandout.pdf

A one-page explanation of the peer review process: Peer review instructions.pdf

Executive summary requirements and marking sheet: Executive Summary Instructions.pdf

 Page source:  Gray, K., Waycott, J., Thompson, C., Clerehan, R., Sheard, J., Hamilton, M., & Richardson, J. (2011) //Using Social Web (Web 2.0) Activities for Student Assessment: Resources for University Learning and Teaching.// Retrieved from https://web2assessmentresources.wikispaces.com