DSC assessment moderation guidelines


These guidelines explain what assessment moderation is and why is it important. The guidelines provide a holistic moderation process complete with examples, recommendations and a checklist for adaptation to your learning and teaching (L&T) context.
Download PDF Version1MB

What is assessment moderation?


A cyclical process in five stages (see figure 1) that assures systematic fairness in assessment by including internal and external processes where multiple stakeholders identify, agree upon, assure and monitor standards (see figure 2).

Image

Figure 1: Phases of the assessment moderation cycle


Download Assessment Moderation ChecklistA guide to key steps in the process (428 KB)

Phase 1: Designing assessment for equity (fairness and consistency)

Focus:  planning for the quality of assessment tasks with reference to course and program learning outcomes and to achieve fairness and consistency.

Points to check:

  • Do all students have equal opportunity to demonstrate learning outcomes?
  • Is there any possible bias or discrimination in the assessment criteria, their possible interpretation or application ?

Processes:  informal or formal peer scrutiny, or review by a committee or professional accreditation body.


Effective Practice : Calibrating assessment tasks (Examinations in the School of Property, Construction and Project Management)

The School of Property, Construction and Project Management (PCPM) has developed rigorous moderation processes and tools to moderate examinations. In collaboration with the responsible program manager, the course coordinator appoints one or more examination moderators to check a proposed exam before it is released. Using a pro forma (refer PCPM Exam Moderation Checklist), the moderator reviews content coverage, technical accuracy, the academic validity of the examination questions and the marking scheme, clarity of questions and instructions and the quality of the layout. The course coordinator actions agreed amendments to create a new moderated version that is then referred to the program manager for approval. Only after associate dean or program manager approval is the exam finalised.

Why is assessment moderation important?


Assessment moderation:

  • Is critical for the development of effective educational practice, including ongoing reflection
  • assures academic quality by verifying the credibility of assessment as evidence of student achievement and the credibility and reliability of assessors’ judgements
  • may support equity
  • may help to meet professional and system requirements

Equivalence and comparability


Where courses are delivered at multiple locations, assessment moderation will require consideration and planning for equivalence and comparability.

Equivalence: maintaining quality standards across multiple offerings in terms of course management, learning outcomes, resources, assessment, teaching, work-integrated learning (WIL) and internationalisation.

Comparability: extent to which offerings have been contextualised and customised to meet local needs, including those of the student cohort, to achieve equivalence.

Contextualisation: Customisation of learning activities, resources, assessment, delivery, teaching practice and student learning support to facilitate learning for a specific student cohort.

See RMIT program and course review process for further information


Who should be involved in moderation?


Course coordinators have principal responsibility for assessment design and quality, including moderation processes, but this responsibility needs to involve multiple stakeholders in ongoing discussions and genuine exchange.

Other stakeholders include teaching teams (local and international) and program advisory committees and may also include professional and industry bodies, relevant community partners and contacts.

Genuine collaboration and reciprocity among teaching teams and other stakeholders is especially important when designing curriculum and assessment in transnational teaching contexts. This means:


  • valuing and drawing upon the specific and contextual professional knowledge and expertise that all members of the teaching team bring
  • using data to learn about the student cohort
  • differentiating core curriculum content from customisable content
  • constructively aligning curriculum
  • being adaptive during delivery
  • evaluating performance data.

Discussions need to happen:


  • when designing assessment (alignment with professional standards, qualification level, learning outcomes and activities), assessment criteria and marking guides
  • through calibration of assessment requirements, standards and achievement and  validation of judgements through ongoing evaluation and adaptation of assessment design and practice.
Image

Figure 2: Key stakeholders in assessment moderation

Recommendations for effective practice


Assessment moderation should involve ongoing dialogue

  • Talk to your teaching team as early as possible about assessment and keep up the conversation during the teaching period.
  • If possible, collaborate on developing assessment criteria, standards and evidence of performance, keeping in mind the need to align assessment criteria, assessment tasks, learning outcomes and pedagogy. This will help all assessors to internalise requirements and standards and inform conversations with students about the assessment requirements.
  • Keep in mind the role of culture in shaping communication, interaction, pedagogy (student and teacher roles, teaching practices) and assessment, including underlying values and assumptions.
  • Be alert to stereotypes and ethnocentricity and remember the importance of context for L&T, particularly for transnational offerings.
Consider developing marking guidelines and exemplars and aim to moderate assignments such as oral presentations in real time.

Involve students in understanding assessment criteria

  • Consider asking students to review each other’s work using developed assessment criteria and/or a marking rubric to help them to internalise standards. This can help students to understand those criteria and develop evaluative judgement about their own performance.

Practical tips

  • Align feedback assessment criteria and performance descriptors so that students can relate comments back to these
  • Make sure that your Part B course guide accurately describes assessment and related administrative matters such as extensions, special consideration and penalties for late submission
  • Clearly define the group component of group assignments (see clause 1.20 RMIT Assessment Processes)
  • Include a clearly defined process for when a group becomes dysfunctional so that academic staff can intervene before assessment submission (see clause 1.18 of RMIT Assessment Processes).

Guiding policy


External Policy

Higher Education Standards Framework Threshold Standards 2015 (Cth) (HESF)

Consistency and validation of assessment (HESF standard 1.4.3)

  • Requirement for consistency between assessment and demonstration of learning outcomes and validation of student attainment and grades awarded.

 

Monitoring and review (HESF standard 5.3)

  • Requirements around monitoring, review and improvement of courses, including assessment moderation processes, such as comprehensive reviews of assessment methods and students’ achievement of learning outcomes (at least every 7 years); interim monitoring of student progress (grade distribution); and regular peer review of student success against comparable programs of study from another provider (student progress, assessment methods, grading with cohort analysis) to ‘calibrate’ assessors’ grading.

RMIT Policy

The RMIT Assessment processes (clauses 3.1-3.130) (clauses 3.1-3.30) set out practices for moderating assessment during the teaching period, such as specification of a marking scheme, allocation of a single marker (and a moderator) and double marking of assessment where there are new markers in a course. To ensure consistency, cross marking and review of assessors’ marking may occur.

Consensus moderation (clause 3.1)

Before results are finalised, Course Assessment Committees (CACs) moderate assessment by checking grade distribution and results across different offerings of the same course.

 

 

Reporting to CAC (clause 3.25.6)

Course coordinators report to the CAC on any issues arising with assessment, reasons for adjustments made to marks, and any other matter relevant to approval of final results Adjustments need to be documented, with reasons included. All narrow fails (45-49) must be double marked (clause 3.7).

CAC moderation (clause 3.23)

  • may lead to adjustment of student results to overcome disparities in the difficulty of assessments and/or severity of marking
  • adjustments must be made before the marks for the assessment are communicated to students where possible (clause 3.23.1).

 

 

Program assessment boards – PABs (clause 3.13)

PABs consider CAC recommendations on student or program cohort performance, how effective program assessment is and whether it needs enhancement.

Equivalence and comparability

RMIT program and course review process includes provisions for assuring equivalence and comparability of courses offered across multiple locations.

Section 24 of the Program and Course Policy states requirements for RMIT programs delivered by partners.