Your Task
The task consists of 15 rounds. In each round, your task is to evaluate one summary.
The summaries were produced by participants in earlier experimental sessions of this study.
{{ if player.treatment == 'AIinfo' }}
In those sessions, participants had access to an embedded Artificial Intelligence (AI) tool
that could assist them in producing their summaries, but using the tool was optional.
{{ endif }}
You will be provided the original text corresponding to the respective summaries.
Your task is designed to resemble real tasks that managers perform at the workplace.
We would like you to take them seriously and treat them as if they are part of your real job.
Grading
You will be asked to evaluate each of the summaries based on the following criteria:
(1) Accuracy:
The summary correctly reflects the key facts, ideas, and intent of the original text.
(2) Clarity:
The summary is easy to understand, well-structured, and expressed in straightforward language,
avoiding unnecessary complexity.
(3) Overall Quality:
The summary effectively captures the most important points in a concise, coherent, and readable way.
You will rate the respective criteria on a 1–3 scale
(1 = “Poor”, 2 = “Satisfactory”, 3 = “Outstanding”).
Payment
You will receive a fixed show-up fee of £2.50. In addition, you can earn a bonus based on your performance.
At the end of the experiment, 3 of these 15 rounds will be randomly selected for payment.
In each round, you will evaluate one summary. Each summary is independently rated by three evaluators
on the following criteria: Accuracy, Clarity, and Overall Quality.
For each criterion, your payment depends on whether your rating matches the majority rating among the
three evaluators, defined as the rating chosen by at least two of the three evaluators.
If all three evaluators provide different ratings, that is, {1, 2, 3}, there is no majority.
In this case, the computer randomly selects one of the three ratings to serve as the majority rating.
For each criterion where your rating matches the majority rating, you earn £0.50.
Therefore, you can earn between £0.00 and £1.50 per selected round:
- 0 matches → £0.00
- 1 match → £0.50
- 2 matches → £1.00
- 3 matches → £1.50