Your Tasks
The experiment contains 15 summarisation tasks that are designed to resemble real tasks that workers
perform at the workplace. We would like you to take them seriously and treat them
as if they are part of your real job.
The experiment will last for 15 rounds. In each round, you will receive a different text, and your task
is to produce a summary of the text presented.
You will have 2 minutes to read the text and produce the summary. Once the 2 minutes have elapsed,
you will be automatically moved to the next round.
Please follow these guidelines carefully when constructing your summaries:
- The total length of your summary must not exceed 500 characters.
- Your summary should capture the main ideas of the text.
- Do not include opinions, interpretations, or information not in the text.
{{ if player.treatment == 'NoAI' }}
{{ else }}
To assist with your summaries, you will be provided with five bullet points generated by an embedded Artificial Intelligence (AI) tool. These bullet points are available in every round, but their use is optional.
{{ endif }}
Before the experiment begins, one practice round will be played, for which you will not be paid.
This is just for you to familiarise yourself with the task. The practice round will last for 2 minutes.
Grading
Your summaries will be graded by independent evaluators. Each evaluator will be given the exact summaries
that you have produced today and will be asked to evaluate based on the following criteria:
(1) Accuracy:
The summary correctly reflects the key facts, ideas, and intent of the original text.
(2) Clarity:
The summary is easy to understand, well-structured, and expressed in straightforward language,
avoiding unnecessary complexity.
(3) Overall Quality:
The summary effectively captures the most important points in a concise, coherent, and readable way.
Independent evaluators will rate the respective criteria on a 1–3 scale
(1 = “Poor”, 2 = “Satisfactory”, 3 = “Outstanding”).
Each summary will receive three independent evaluations, and its final rating will be the average
of the respective ratings across the three evaluators.
Payment
You will receive a fixed show-up fee of £2.50. In addition, you can earn a bonus based on your performance.
At the end of the experiment, 5 of the 15 rounds will be randomly selected for payment.
Your total payment is the sum of your earnings from the 5 randomly selected rounds.
The following table shows the payment corresponding to your average final rating per summary:
|
Average final rating = X
|
Final payment
|
| 1.0 <= X < 1.5 |
£0.00 |
| 1.5 <= X < 2.5 |
£1.50 |
| 2.5 <= X <= 3.0 |
£3.00 |
For example:
Suppose that the first evaluator assigned a rating of 3 for Accuracy, 2 for Clarity, and 3 for Overall Quality.
The second evaluator assigned a rating of 3 for Accuracy, 2 for Clarity, and 2 for Overall Quality.
The third evaluator assigned a rating of 2 for Accuracy, 2 for Clarity, and 1 for Overall Quality.
Therefore, your average rating for each of the criteria will be:
- (3 + 3 + 2) / 3 = 2.67 for Accuracy
- (2 + 2 + 2) / 3 = 2 for Clarity
- (3 + 2 + 1) / 3 = 2 for Overall Quality
Your final rating for this summary = (2.67 + 2 + 2) / 3 = 2.22
Your final payment for this summary will therefore be £1.50.