The experiment contains 15 summarisation tasks that are designed to resemble real tasks that workers perform at the workplace. We would like you to take them seriously and treat them as if they are part of your real job.
The experiment will last for 15 rounds. In each round, you will receive a different text, and your task is to produce a summary of the text presented.
You will have 2 minutes to read the text and produce the summary. Once the 2 minutes have elapsed, you will be automatically moved to the next round.
Please follow these guidelines carefully when constructing your summaries:
To assist with your summaries, you will have access to an embedded Artificial Intelligence (AI) tool. This tool is available in every round, but using it is optional.
{{ endif }}Before the experiment begins, one practice round will be played, for which you will not be paid. This is just for you to familiarise yourself with the task. The practice round will last for 2 minutes.
Your summaries will be graded by independent evaluators. Each evaluator will be given the exact summaries that you have produced today and will be asked to evaluate based on the following criteria:
(1) Accuracy: The summary correctly reflects the key facts, ideas, and intent of the original text.
(2) Clarity: The summary is easy to understand, well-structured, and expressed in straightforward language, avoiding unnecessary complexity.
(3) Overall Quality: The summary effectively captures the most important points in a concise, coherent, and readable way.
Independent evaluators will rate the respective criteria on a 1–3 scale (1 = “Poor”, 2 = “Satisfactory”, 3 = “Outstanding”).
Each summary will receive three independent evaluations, and its final rating will be the average of the respective ratings across the three evaluators.
You will receive a fixed show-up fee of £2.50. In addition, you can earn a bonus based on your performance. At the end of the experiment, 5 of the 15 rounds will be randomly selected for payment.
Your total payment is the sum of your earnings from the 5 randomly selected rounds.
The following table shows the payment corresponding to your average final rating per summary:
| Average final rating = X | Final payment |
|---|---|
| 1.0 <= X < 1.5 | £0.00 |
| 1.5 <= X < 2.5 | £1.50 |
| 2.5 <= X <= 3.0 | £3.00 |
For example:
Suppose that the first evaluator assigned a rating of 3 for Accuracy, 2 for Clarity, and 3 for Overall Quality. The second evaluator assigned a rating of 3 for Accuracy, 2 for Clarity, and 2 for Overall Quality. The third evaluator assigned a rating of 2 for Accuracy, 2 for Clarity, and 1 for Overall Quality.
Therefore, your average rating for each of the criteria will be:
Your average final rating for this summary = (2.67 + 2 + 2) / 3 = 2.22
Your final payment for this summary will therefore be £1.50.