Skip to content
English
Visit edly.io
  • There are no suggestions because the search field is empty.

How Open Response Assessment Problem Type Functions?

How Open Response Assessment (ORA) Works in Open edX

Overview

An Open Response Assessment (ORA) allows learners to submit open-ended text answers, short essays, or uploaded file attachments. Following submission, learners complete a structured evaluation workflow—which can include training modules, grading their peers, evaluating their own work, or undergoing formal grading by the course team—to complete the activity.

Tip: Ensure you are logged into Open edX Studio before proceeding.

Note on Cohorts: By default, ORA components do not restrict peer assessments by cohort boundaries—a learner in one cohort may be assigned to review a submission from a learner in a different cohort. To limit peer assessment to learners within the same cohort, place the ORA component inside a unit whose visibility is restricted to that cohort using content group or enrollment track settings.

Core Structural Elements of an ORA

1. Prompts (Questions)

The prompt represents the core task or inquiry presented to the learner. An individual ORA block can contain a single prompt or multiple distinct questions.

  • In the configuration area, write clear guidelines specifying preferred word counts, scope parameters, or file formatting instructions.
  • File Submissions: You can toggle settings to allow or mandate that learners upload images, code packages, or PDF documents alongside their text response.

2. The Assessment Rubric

A unified rubric is used to evaluate all responses within an individual ORA component. The grading template is organized around custom evaluation benchmarks:

  • Criteria: Defines a specific dimension of the response to be judged (e.g., Technical Accuracy or Structural Organization). Each criterion has a unique identifier name and a descriptive prompt explaining what an evaluator should look for.
  • Options: Represents the sliding scale of performance metrics within a criterion (e.g., Poor, Satisfactory, Excellent). Each individual option requires a unique name, an actionable explanatory note helping the grader select accurately, and a point value.

Note: Different criteria blocks within the same rubric can house varying numbers of point choices. You can also deploy feedback-only criteria that contain no points but offer a blank text field for open commentary.

Managing the Assessment Steps Workflow

The progression flow of the assessment is dictated by the chronological steps toggled on within the component settings.

Step 1: Learner Training Step

This step functions as an onboarding exercise to align learners with grading expectations before they evaluate actual student submissions.

  1. In the component editor, enable Learner Training.
  2. Procedures: You must provide one or more sample responses along with your definitive instructor score selections for each option.
  3. Learner Interaction: In the LMS, the learner is shown your sample text and must use the rubric to match your intended score. The system will prompt them to adjust their grading until their option choices perfectly align with the instructor template before unlocking subsequent steps.

Rule: If you toggle on Learner Training, you must also enable Peer Assessment. The training step must chronologically precede peer or self-evaluation cycles.

Step 2: Peer Assessment Step

In this step, learners anonymously evaluate a set number of submissions from other learners enrolled in the course.

  1. Scroll to the Peer Assessment settings panel in the ORA editor.
  2. Set the minimum number of submissions each learner must review to complete the step (e.g., 3), and set the minimum number of peer reviews each submission must receive before its peer score is finalized.

Step 3: Self Assessment Step

This step requires learners to objectively critique their own work against the established rubric.

  1. Locate and toggle on the Self Assessment step.
  2. In the LMS, the learner is shown a copy of their original text submission alongside the interactive rubric blocks to complete their self-grade.

Step 4: Staff Assessment Step

This step designates that an official member of the course team must grade the submission.

  1. Locate the Staff Assessment step block at the bottom of the ORA editor.
  2. Procedures: To grade submissions, go to the Instructor Dashboard in the LMS, navigate to the ORA component, and score each submission using the rubric.

Rule: When Staff Assessment is enabled, the final grade remains locked until a staff member enters a score. The instructor's rubric selection overrides all other evaluation types, including pre-existing peer or self-assessment marks.

Best Practices for Course Teams

  • Sequential Arrangement: The platform enforces a fixed step order: Learner Training → Peer Assessment → Self Assessment → Staff Assessment. You do not need to reorder steps manually; simply enable the steps you want to use.
  • Rubric Integrity: Once an ORA assignment is published and a single learner submits an answer, the Criterion Names and core structural weights lock completely on the database. Ensure your rubric text, spelling, and scoring options are fully vetted before publishing.