Skip to content
English
Visit edly.io
  • There are no suggestions because the search field is empty.

How Open Response Assessment Problem Type Functions?

How Open Response Assessment (ORA) Works in Open edX

Overview

An Open Response Assessment (ORA) allows learners to submit open-ended text answers, short essays, or uploaded file attachments. Following submission, learners complete a structured evaluation workflow—which can include training modules, grading their peers, evaluating their own work, or undergoing formal grading by the course team—to complete the activity.

Tip: Ensure you are logged into Open edX Studio before proceeding.

Note on Cohorts: By default, global ORA blocks do not restrict peer assessments by cohort boundaries; a learner in Group A might be assigned to review a submission from a learner in Group B. To isolate ORA workflows strictly within defined cohorts, you must place the ORA component inside a unit gated by access-control visibility rules.

Core Structural Elements of an ORA

1. Prompts (Questions)

The prompt represents the core task or inquiry presented to the learner. An individual ORA block can contain a single prompt or multiple distinct questions.

  • In the configuration area, write clear guidelines specifying preferred word counts, scope parameters, or file formatting instructions.
  • File Submissions: You can toggle settings to allow or mandate that learners upload images, code packages, or PDF documents alongside their text response.

2. The Assessment Rubric

A unified rubric is used to evaluate all responses within an individual ORA component. The grading template is organized around custom evaluation benchmarks:

  • Criteria: Defines a specific dimension of the response to be judged (e.g., Technical Accuracy or Structural Organization). Each criterion has a unique identifier name and a descriptive prompt explaining what an evaluator should look for.
  • Options: Represents the sliding scale of performance metrics within a criterion (e.g., Poor, Satisfactory, Excellent). Each individual option requires a unique name, an actionable explanatory note helping the grader select accurately, and a point value.

Note: Different criteria blocks within the same rubric can house varying numbers of point choices. You can also deploy feedback-only criteria that contain no points but offer a blank text field for open commentary.

Managing the Assessment Steps Workflow

The progression flow of the assessment is dictated by the chronological steps toggled on within the component settings.

Step 1: Learner Training Step

This step functions as an onboarding exercise to align learners with grading expectations before they evaluate actual student submissions.

  1. In the component editor, enable Learner Training.
  2. Procedures: You must provide one or more sample responses along with your definitive instructor score selections for each option.
  3. Learner Interaction: In the LMS, the learner is shown your sample text and must use the rubric to match your intended score. The system will prompt them to adjust their grading until their option choices perfectly align with the instructor template before unlocking subsequent steps.

Rule: If you toggle on Learner Training, you must also enable Peer Assessment. The training step must chronologically precede peer or self-evaluation cycles.

Step 2: Peer Assessment Step

In this step, learners anonymized within the platform evaluate a predetermined number of submissions uploaded by other students in the course run.

  1. Scroll to the Peer Assessment settings panel in the ORA editor.
  2. Define the minimum number of submissions a student must grade to complete the step (e.g., 3), and set the buffer number of reviews an individual file must receive to finalize its peer score.

Step 3: Self Assessment Step

This step requires learners to objectively critique their own work against the established rubric.

  1. Locate and toggle on the Self Assessment step.
  2. In the LMS, the learner is shown a copy of their original text submission alongside the interactive rubric blocks to complete their self-grade.

Step 4: Staff Assessment Step

This step designates that an official member of the course team must grade the submission.

  1. Locate the Staff Assessment step block at the bottom of the ORA editor.
  2. Procedures: Instructors access submissions directly from the instructor panel in the live LMS view of the module to grade them using the rubric.

Rule: When Staff Assessment is enabled, the final grade remains locked until a staff member enters a score. The instructor's rubric selection overrides all other evaluation types, including pre-existing peer or self-assessment marks.

Best Practices for Course Teams

  • Sequential Arrangement: Arrange your enabled steps in a logical sequence. The platform enforces that Learner Training happens first, followed by Peer Assessment, Self Assessment, and terminates with Staff Assessment.
  • Rubric Integrity: Once an ORA assignment is published and a single learner submits an answer, the Criterion Names and core structural weights lock completely on the database. Ensure your rubric text, spelling, and scoring options are fully vetted before publishing.