GroundTruth text labelling - hide data columns, and methods of quality control

1

I have a csv of sentences which I'd like labelled, and have identified GroundTruth labelling jobs as a way to do this. Having spent some time exploring the service, I have some questions:

**1) **I can't find a way to display only particular columns to the labellers - e.g. if the dataset has a column of IDs for each sentence, this ideally shouldn't be shown to labellers

2) There is either single labelling or multi labelling, but I would like a way to have two sets of single-selection labels, where one captures difficulty of assigning the label:

Select one for binary classification a) Yes, b) No

Select one for difficulty of classification c) Easy, d) Medium, e) Hard

Can this be done using custom HTML? Is there a guide to writing this - the template it gives you doesn't seem to render as-is.

3) There appears to be a maximum of $1.20 payment per task. Is this the case, and why?

4) Having not used mechanical turk before, are there ways of ensuring people take the work seriously and don't just select random answers? I can see there's an option to have x number of people answer the same question, but is there also a way to put in unambiguous questions to which we already have a 'pre_agreed_label' every nth question, and remove people from the task if they get them wrong?

Thanks!

  • Did you eventually find an answer yourself for number 4?

1 回答
0

Good question , did you find the answers about this as i'm curious too

已回答 1 年前

您未登录。 登录 发布回答。

一个好的回答可以清楚地解答问题和提供建设性反馈,并能促进提问者的职业发展。

回答问题的准则