Handbook
Complete edition
EthelExam is the platform course staff use to view, score, and finalize AI-scored exams, and students use to view their results and raise objections ("vetoes"). It is part of the Ethel project at ETH Zürich. The reference instance, serving ETH Zürich courses, runs at exam.ethel.ethz.ch.
EthelExam is the visible end of a service that accompanies the whole examination process. The Ethel team supports course teams in designing the exam and its rubrics so they work well with AI scoring, assists with scanning the completed exams, runs the AI scoring of every problem part, and imports and provisions the results in EthelExam — where course staff verify the scores and students view their results and submit vetoes. Manual scoring is needed only where a veto asks for it, and the examiner always retains final grade authority.
This service is developed in the exam-grading-assistance project STIFT+, funded by the Rector's Impulse Fund 2026 (project description, PDF): an initiative to keep open-ended, handwritten STEM exams viable at scale by combining multimodal AI scoring with human oversight. Participating courses receive central support for assessment design, scanning, AI integration, and data handling. Participation is an R&D collaboration between the course team and the Ethel team.
Audience: course admins, teaching assistants (TAs), and students.
This handbook has five shared parts and one part per role. Read Key concepts and Getting started first, then the part for your role.
| Part | For |
|---|---|
| Key concepts | everyone |
| Getting started | everyone |
| Course admin guide | faculty running an exam |
| TA guide | teaching assistants scoring vetoes |
| Student guide | exam participants |
| Features in evaluation | everyone |
| Troubleshooting | everyone |
| Glossary | everyone |
EthelExam is run by the Ethel team, the group that operates the platform and provides support. Whenever this handbook says to contact the Ethel team, write to:
Include the exam (course code and term), the problem part if one is affected, what you expected, what you saw instead, and roughly when it happened. The team also appreciates hearing your feedback. Questions about a score itself are a matter for the course, not the Ethel team.
All names, exams, and documents shown in screenshots are synthetic example data.
| Version | Date | Changes |
|---|---|---|
| 1.0 (draft) | 21 August 2026 | First edition. 20260822-041925-b6b27b7f |
Provided by the Ethel team, ETH Zürich.
This chapter explains the vocabulary and the scoring model that everything else in EthelExam builds on. It is short. Read it once and the rest of the handbook will make much more sense.
An exam in EthelExam is one course's examination in one term, for example Physics I, PHYS-101, FS 2026. Inside an exam:
For each problem part a student answered, EthelExam stores the scanned document with that answer (a PDF). It is shown right next to the scores, so the score can always be checked against the actual work. Parts without an answer are marked absent.
Every problem part has a rubric: a list of criteria, each worth a number of points. A score in EthelExam is not a single number. It is one value per rubric criterion, and the total follows from them.
Scores come from two sources:
During normal scoring in EthelExam, score changes are never destructive: every scoring action is recorded, and course admins can inspect the full score history of any problem part.
A veto is an objection against the current score of one problem part: "please look at this again". Vetoes drive the whole scoring workflow.
Two time windows, set by the course admin, control what students can do:
There is no separate "publish results" step. As soon as the viewing period is active, students see the current scores of their problem parts, except parts with an open veto, whose scores stay hidden until they are scored again.
Access is per exam: the same person can be a course admin in one exam, a TA in another, and a student in a third. Within one exam:
| What you can do | Course admin | TA | Student |
|---|---|---|---|
| See student names and Legi | yes | no — pseudonymous labels | own data only |
| See students' documents | all | all, without names | own only |
| See AI scores | always, including history | never | own, when the viewing period is active and the part has no open veto |
| See manual scores | always, including history | on manually scored parts | own, under the same conditions |
| Enter scores | on vetoed or manually scored parts | on vetoed or manually scored parts | never |
| Create vetoes | any eligible part, including in bulk | no | own parts, during the veto period |
| Edit periods | yes | no | no |
| See scoring notes | all | only on parts they can score | never |
Three of these rules are worth explaining:
A scoring note is a private staff notepad on one problem part of one student, for example "partial credit discussed with lecturer, see 3.c". Notes are visible to course staff only, never to students. Course admins see every note including who wrote it; TAs can only read and write notes on parts they are allowed to score, and see notes without author names.
Importing exam data, resetting an exam, and managing who is enrolled are operational tasks performed by the Ethel team on request. They are not self-service functions in the web interface. If data looks wrong or missing, contact the Ethel team (see Getting help) rather than looking for an import button.
Course teams that run the scanning and AI scoring themselves can prepare an import package and hand it to the Ethel team for import — see the EthelExam Import Package Guide, available from the Ethel team. This must be arranged well in advance of the exam.
EthelExam uses Switch edu-ID, the same account you use for other Swiss university services. There is no separate EthelExam password.
Open the EthelExam address for your institution — at ETH Zürich this is exam.ethel.ethz.ch — and select edu-ID Sign-in. You are taken to the Switch edu-ID login and, after signing in, back to EthelExam.
The first time you sign in, EthelExam shows you exactly which information it received from your Switch edu-ID account: your name, email address, and your university affiliations. Review the list and select Agree and continue. This appears only once; afterwards you go straight to your exams.
After signing in you land on Exams: every exam you have a role in, one card per exam. The card shows the exam's viewing and veto periods, and the actions your role allows. A student sees a View button; a TA also sees Vetoes; a course admin sees View, Vetoes, Exam overview, and Exam settings.
If an exam you expect is missing, the most common cause is that your university affiliation is not linked to your edu-ID account. The Missing an exam? panel at the bottom of the page explains how to check and fix this. If the exam is still missing afterwards, contact the Ethel team (see Getting help).
The header is the same on every page: breadcrumbs on the left show where you are (exam name, current page), followed by the navigation buttons your role allows. Where a button carries a badge, the number is the count of open vetoes.
Page addresses are stable: a bookmark or a shared link lands on the same page again, provided the person opening it has a role that allows it.
As a course admin you configure the exam's periods, monitor scoring progress, view any student's work with full identity, correct scores, and manage the veto workflow. This chapter follows the life of an exam from setup to finished scores.
On the Exams page, your exam card carries an Exam settings button. It opens the period editor in place:
Remember: there is no "publish results" button. Setting the viewing period is the release. When it opens, students see their scores, except parts with an open veto.
Exam overview is your progress dashboard: one row per student, one column per problem part, with the summary strip on top (submitted and absent parts, AI-scored and manually scored counts, total and open vetoes).
Reading the grid:
Above the grid:
Search filters by student name or Legi as you type.
Filter narrows the grid to students with open vetoes and/or manually scored parts.
With both filters active, the grid narrows to the students who still need attention:
Download CSV exports the full grid (always the complete data, not the filtered view) as a spreadsheet-ready file. Values open cleanly in Excel and LibreOffice, including names with special characters.
Refresh re-reads the grid without leaving the page, which is useful while TAs are actively scoring.
For very large exams (beyond roughly 50,000 grid cells) the grid view is not available and the page says so; the rest of the app works normally.
From the overview, or via View, you see one student's entire exam: every problem part with its rubric scores on the left and the student's scanned answer on the right. As a course admin you see the student's real name and Legi.
Things only course admins see here:
The AI delta: where a manual score deviates from the AI baseline, the point difference is shown next to the affected criteria.
The History button: the complete score history of the part, listing every AI and manual scoring event with its timestamp, actor, and per-criterion values.
Student exam and Rubric links above the document: the first opens this student's exam in a new tab, the second opens the official scoring rubric for the problem part.
Who may enter scores on a problem part depends on its state:
While you edit, your changes are drafts until you press Score; the status icon next to the buttons shows whether the row is saved. If someone else scores the same part while you have unsaved edits, neither version is discarded. The card shows you both, and you choose Use saved scores or Keep my version.
Three ways, increasing in scope:
One problem part: the Veto button on the part's card — on a student's View page or in the problem part view. A confirmation dialog reminds you that the veto is final before you submit:
Before the veto, the part shows its current score:
After submitting, the score is void and the card is in edit mode, ready to be scored again by you or by a TA:
All parts of one student: on a student's View page, Veto all problem parts for this student. A confirmation dialog with an explicit checkbox stands between you and the action:
One part across all students: on the problem part view (see One problem part across all students), Veto problem part for all students, with the same kind of confirmation.
Bulk vetoes skip parts that cannot be vetoed (already manually scored, or already vetoed). Remember that vetoes are permanent: each of these rows will need a manual score before its result is visible to the student again.
Once a part has an open veto, a TA finds it in the veto worklist and in the student's exam as a blind, zeroed card, ready to be scored:
The student's own view hides the score entirely until the part has been scored again:
Clicking a problem-part header in the overview opens the problem part view: the same part for every student, one card per student, with documents. Use it when a systematic issue surfaces, such as a misprinted sub-question or an unexpected alternative solution path, and you want to sweep through all answers to that one part.
Vetoes shows the state of the veto queue per problem part: how many vetoes each part has and how many are still open.
Clicking a part opens its veto scoring worklist: every vetoed instance of that part, open vetoes first, each with the student's document. Enter the rubric values and press Score; that closes the veto in the same step.
TAs work the same worklist, but blind: where you see the current scores, a TA sees a zeroed rubric, so their fresh scoring is not anchored by the disputed number. The student sees no score for the part until it has been scored again.
The summary strip and the Vetoes badge track your progress. The exam is done when open vetoes reach zero and stay there after the veto period closes.
Every problem part card has a Scoring note section: a staff-only notepad for that student's part. Use it for anything a colleague picking up the same part should know. Notes show author and timestamp to course admins; students never see notes.
Notes keep their full version history. The clock icon opens Previous notes, where earlier versions can be reviewed and restored:
Imports, re-imports, exam resets, and enrolment changes are handled by the Ethel team (see Key concepts). Be aware that a re-import starts from scratch: the exam is fully reset, and all work done in EthelExam — vetoes, manual scores, scoring notes — is lost. Plan for the lead time: if something in the underlying data must change during the scoring window, contact the Ethel team as early as possible. A re-import comes with a scoring freeze, announced to all staff.
As a TA you score vetoed problem parts again. You see the students' work and documents, but never their identities, and on parts with an open veto you score blind. Both are by design, and both are explained below.
Your exam card shows a Vetoes button with the queue state. On the exam card, "8 / 20" means 8 of 20 vetoes have been scored. That button is your main entry point.
In the veto worklists, students appear as neutral five-character identifiers (pseudonymous labels) such as "0BASP" instead of names. The label does not reveal the student's identity to you, and it is only valid within one problem part: the same student carries a different identifier on each part of the exam, and the whole-exam view shows no identifier at all. You judge each answer on its own; this pseudonymity is what lets a student trust that a veto was judged on the work.
Your view of any problem part depends on its state:
Vetoes lists every problem part with its veto counts:
Pick a part with open vetoes and you land in its veto scoring worklist: one card per vetoed answer, the scanned document on the right, the rubric on the left.
On a card with an open veto, every rubric value shows as zero and the card is already in edit mode. This is intentional: blind scoring. You are not shown the AI's scores or any previous manual scores for the disputed part, so nothing anchors your judgment. Read the student's answer, check it against the rubric (the Rubric link opens the official rubric excerpt for this part), fill in every criterion, and press Score.
Two things happen when you score an open veto:
The veto closes. Scoring is the resolution; there is no separate accept or reject step.
The card moves down the list on the next refresh, so open work stays on top:
Cards whose veto is already closed show the current scores, and you can still correct them.
While a veto is open, the student sees no score for that part — only a notice that it will be scored again. The moment you press Score, the new result becomes the student's score:
From a veto card, Student exam opens that student's entire exam, still without any name — the exam view shows no identifier at all. Use it when an answer refers to another problem part or continues on a different page: answers sometimes depend on each other, and the whole exam gives you the context. The visibility rules above follow you: open-veto parts are blind and editable, manually scored parts show their scores, untouched parts show their documents but no scores.
On parts you can score, the Scoring note section lets you read and leave staff-only notes. You see notes without author names (only whether a note is your own); course admins see all notes with authors. Students never see notes.
Notes are versioned: every edit is recorded, and the clock icon opens Previous notes with the earlier versions. Nothing you write or overwrite is lost.
Several people can work the same queue. If someone scores a part you are editing, your card offers Use saved scores or Keep my version; nothing is lost silently. If you leave a page with unsaved edits, the app warns you first.
As a student you can view your scored exam during the viewing period and check every score against your answer sheets. During the veto period you can also ask for individual problem parts to be scored again.
After signing in you see your exams. Each card shows the viewing period (when you can look at your exam) and the veto period (when you can object to scores). The View button opens your exam; if you have vetoes still open, a badge on the button shows how many.
If an exam you took is missing here, see Missing an exam? at the bottom of the page; usually your university affiliation needs to be linked in your edu-ID account.
Your exam is shown problem by problem: for each problem part, the scoring rubric with your points on the left, and the scan of your answer on the right, so you can check every point against what you actually wrote. The viewer zooms, and you can open any document in its own tab.
Before the viewing period starts (or after it ends) the page shows a notice instead of your exam:
If you believe a problem part was scored wrongly, you can veto it. This asks course staff to score that part again from scratch.
Press Veto on the problem part. A confirmation dialog spells out the rules before you commit:
Read the dialog carefully; the veto is deliberately final:
As soon as your veto is submitted, the part shows an open-veto notice and its rubric and scores disappear from your view. That is intentional: the previous score is void, a fresh scoring is underway, and its outcome is what counts.
When course staff have scored the part again, the veto closes automatically and the part shows its new score. The part is then final: it can no longer be vetoed.
A veto submitted before the period closes is still processed after it. If the viewing period is still active, you will see the new score as soon as staff have entered it. Outside the viewing period the exam is no longer visible in EthelExam; contact your course if you need the result. When a part is final, EthelExam has no further objection path.
Some EthelExam features are being piloted with selected courses before they become generally available. They are described here rather than in the role chapters. Whether a feature is active in your exam depends on its configuration, and details may still change.
When this feature is enabled for an exam, a student submitting a veto can attach a short written explanation pointing staff at what they should look at ("the substitution in line 3 is correct, see the boxed result").
Course staff who may read explanations see the text on the part's veto card, above the scoring note:
How it behaves:
A problem part can have several rubric variants: alternative scoring rubrics for the same sub-question, covering different valid solution approaches (for example a direct calculation versus an energy argument). The AI scoring may score one or more variants for an answer; course staff then choose which variant becomes the official score for that student's part. Score history is preserved per variant, and all views show the currently selected variant.
After an hour of inactivity (or 8 hours after signing in, whichever comes first) EthelExam ends your session and shows "Your session has expired" on the sign-in page:
Sign in again and you will be returned to the page you were on. Unsaved edits do not survive the expiry; save as you go.
Your exam list shows exams in which your signed-in identity has a role. The usual cause for a missing exam is an unlinked university affiliation in your Switch edu-ID account: open Missing an exam? at the bottom of the exams page, follow the guide, then sign out and back in. Still missing? Contact the Ethel team (see Getting help).
If a scanned answer shows "Document temporarily unavailable", use Retry; this is almost always transient. Open in new tab is a second path to the same document. If a document is persistently unavailable, report the exam, problem part, and time to the Ethel team.
Two staff members had the same problem part open, and the other one saved first. Nothing is lost: the card shows both versions and asks you to pick Use saved scores (accept your colleague's result) or Keep my version (submit yours on top). Coordinate in the veto worklist by working top-down; scored cards drop down the list on refresh.
The overview grid caps out at very large exams (roughly 50,000 cells, which is the number of students multiplied by the number of problem parts). All other pages work normally: use the problem part view and the veto worklists, and contact the Ethel team if you need a grid export for an exam of this size.
You have score or note edits that have not been saved. Stay returns you to the page so you can press Score or Save note; Leave without saving discards the drafts.
For anything data-related — wrong or missing students, wrong documents, re-imports — contact the Ethel team (see Getting help). For access problems, start with An exam is missing from my list above. Questions about a score itself are a matter for the course.
AI scored — The initial state of every problem part: scores from the Ethel team's AI scoring, imported into EthelExam, before any human has intervened.
Blind scoring — What a TA does on a part with an open veto: the disputed score is hidden and the rubric starts at zero, so the fresh scoring is not anchored by the old number.
Course admin — The faculty role that runs an exam: sets periods, sees identities and score history, creates vetoes, and corrects scores.
Document — The scanned PDF of one student's answer to one problem part, shown next to the rubric.
Ethel team — The group that operates EthelExam, provides support, and appreciates hearing your feedback. Contact: ethel@sl.ethz.ch (see Getting help).
Exam — One course's examination in one term, for example Physics I, PHYS-101, FS 2026.
Exam instance — One student's personal copy of an exam, with one row per problem part.
Legi — The Swiss matriculation number identifying a student.
Manually scored — The state of a problem part after a member of course staff has entered scores by hand. A manually scored part can never be vetoed again.
Optional explanation — (in evaluation) The short text a student may attach to a veto, explaining what staff should look at. Visible to course staff only, and never editable after submission. See Features in evaluation.
Problem — A top-level exercise of an exam ("Problem 1 — Rolling sphere").
Problem part — One sub-question of a problem ("1.a — Speed at the bottom of the incline"), and the unit that documents, scores, vetoes, and notes attach to.
Problem part view — The course-admin page that shows one problem part across all students.
Rubric — The list of criteria, each worth points, against which a problem part is scored.
Scoring note — A private staff notepad on one problem part of one student. Versioned; never visible to students.
Student label — The pseudonymous five-character identifier (such as "0BASP") that identifies a student to a TA within one problem part, shown in veto scoring only. It does not reveal the student's identity to TAs; identifiers differ from part to part by design.
TA — The teaching-assistant role: scores vetoed parts blind and never sees student identities.
Veto — A final, one-time objection against the current score of one problem part. Open until staff enter a manual score, which closes it.
Veto period — The window during which students can submit vetoes. Always lies inside the viewing period.
Rubric variant — (in evaluation) One of several scoring rubrics for the same problem part, covering different valid solution approaches. See Features in evaluation.
Veto scoring — The staff workflow (and page) for scoring vetoed parts.
Viewing period — The window during which students can open their exam and see documents and scores.