Questions tagged with Amazon Mechanical Turk

Hi, I fielded a batch on the UI, and am seeing that 90 people responded. But when I pull the HIT table using the below python code, the number of people that it says completed the HIT are 0? Why would that be? Screenshot of the .csv pulled from the Python API vs the screenshot from the UI is attached. I'd really appreciate any help! Python code (with sensitive info censored): ################ # SETUP ################## get_ipython().system('pip install boto3') import boto3 as bt import functools as fn import pandas as pd import numpy as np import datetime as dt import xml.etree.ElementTree as ET import math as m import csv import os import time #Switch to this version when doing the actual fielding mturk = bt.client('mturk', aws_access_key_id = "ZZZZZZZZZZZZz", aws_secret_access_key = "ZZZZZZZZZZZ", region_name='us-east-1', ) # #Creates a table of all HITs PAGE_SIZE = 100 #Must be <= 100 all_hits = mturk.list_hits(MaxResults = PAGE_SIZE) next_token = all_hits["NextToken"] hit_table = pd.DataFrame(all_hits['HITs']) while (all_hits['NumResults'] == PAGE_SIZE): print("Using next token: {}".format(next_token)) all_hits = mturk.list_hits( MaxResults=PAGE_SIZE, NextToken=next_token) hit_table = pd.concat([pd.DataFrame(all_hits['HITs']), hit_table], ignore_index=True) next_token = all_hits['NextToken'] hit_table.to_csv('test.csv'))lg...

When using the HTML code found in the AWS SageMaker docs for the crowd-entity-annotation UI, I found that the demo code they provide (in https://docs.aws.amazon.com/sagemaker/latest/dg/sms-ui-template-crowd-entity-annotation.html) does not work when creating a layout in MTurk Requester Sandbox. It only works when I add this line: ``` <input type="hidden" name="fake" /> ``` outside of the `</crowd-entity-annotation>` element. However, in the preview, and when I publish the HIT, the UI has two submit buttons, one orange and one gray:  A disclaimer needed to be added to make sure the workers did not press the wrong button. This is confusing and leads to poor quality data. How can I correct this? Full code: ``` <script src="https://assets.crowd.aws/crowd-html-elements.js"></script> <crowd-entity-annotation name="crowd-entity-annotation" header="Highlight parts of the text below" labels="[{'label': 'person', 'shortDisplayName': 'per', 'fullDisplayName': 'Person'}, {'label': 'date', 'shortDisplayName': 'dat', 'fullDisplayName': 'Date'}, {'label': 'company', 'shortDisplayName': 'com', 'fullDisplayName': 'Company'}]" text="Amazon SageMaker Ground Truth helps you build highly accurate training datasets for machine learning quickly." > <full-instructions header="Named entity recognition instructions"> <ol> <li><strong>Read</strong> the text carefully.</li> <li><strong>Highlight</strong> words, phrases, or sections of the text.</li> <li><strong>Choose</strong> the label that best matches what you have highlighted.</li> <li>To <strong>change</strong> a label, choose highlighted text and select a new label.</li> <li>To <strong>remove</strong> a label from highlighted text, choose the X next to the abbreviated label name on the highlighted text.</li> <li>You can select all of a previously highlighted text, but not a portion of it.</li> </ol> </full-instructions> <short-instructions> Apply labels to words or phrases. </short-instructions> <div id="additionalQuestions" style="margin-top: 20px"> <h3> What is the overall subject of this text? </h3> <crowd-radio-group> <crowd-radio-button name="tech" value="tech">Technology</crowd-radio-button> <crowd-radio-button name="politics" value="politics">Politics</crowd-radio-button> </crowd-radio-group> <h2> Press the orange colored submit button when you are done! Do NOT press the gray colored submit button! </h2> </div> </crowd-entity-annotation> <input type="hidden" name="fake" /> <script> document.addEventListener('all-crowd-elements-ready', () => { document .querySelector('crowd-entity-annotation') .shadowRoot .querySelector('crowd-form') .form .appendChild(additionalQuestions); }); </script> ```lg...

Hi, I just received the second email from a person who's asking for money because they have completed my sandbox mturk task. They've included their bank details in the message. Is it possible to remove my contact details from sandbox tasks? that's annoying.lg...

If I go to the "Manage" section of MTurk and look at my published batch, it says "Note: If you have edited the Project after publishing this Batch, you will see the latest version."  If I made any changes to my project, such as updating the keywords or changing requirements, do I have to cancel the currently published batch and then republish or can I just leave the published batch as it is?lg...

# Facing an anomaly with my HIT. ## User was able to work on HIT after HIT expiry(LifetimeInSeconds) Sorry if the question seems a bit naive. I am facing an issue with my HIT. I have tested on the sandbox, and it's not possible for workers to accept the HIT after expiration. When a worker previews an assignment just before the expiry time, they can preview it even after the expiry time but they cannot accept it then. But I have got an anomaly. There is one user whose start time for data logged is after 22nd minute of HIT creation where as HIT expires in 5 minutes. I have employed various method to ensure that reload doesn't restarts the survey and survey autosubmits at various stages within first 8 minutes. in short, it's not possible to start the survey after 22nd minute of HIT creation. Thus it only points out that the worker was able to accept the assignment even after expiry. Given that it's not possible on sandbox, I don't think it could be possible on mechanical turk main. But I don't have a production account for mturk worker and thus I can't verify if a worker can somehow accept the task even after the expiry time or not. Does anyone know any case where worker can start work even after expiry?lg...

I am trying to use the [crowd-instance-segmentation](https://docs.aws.amazon.com/sagemaker/latest/dg/sms-ui-template-crowd-instance-segmentation.html) widget for getting instance segmentations. Users can add additional instances of a label using "Add instance." However, if the user selects too many instances, how can instances be removed? There seems to be no functionality for removing extraneous instances. Here is the CodePen corresponding to this widget: https://codepen.io/sagemaker_crowd_html_elements/pen/PoWwvwGlg...

Hello, I have read an instruction and followed all the way it guides. However, my google form link is not working with MTurk with this message: "Layout does not contain any fields for Workers to provide responses. Please include at least one field (input, select, or textarea)" Could you help me address this issue? Thank you.lg...

Hello, I am trying to create a study that opens in the mturk webpage instead of a new window. In the study we are using webcam and I got this message when publish the study in sandbox (so I could try it out). Warning message says: Unable to connect to your webcam, please try again. Perhaps it is being used by another application. So what should I edit in the HTML format so I can give access to webcam using the same page? Is that the solution or should I try to search something else?lg...

I am getting an error saying that I have exceeded monthly credit limit, and cannot publish any batch. I have searched the problem for a while but cannot get a proper solution. Can anyone tell me how to deal with it?lg...

Hi, I get the following error when I try to login to my MTurk requester account with my AWS credentials (as an IAM user): "✖Sorry! Unable to log in We currently do not support the country you are logging in from. We're working hard to make MTurk available for more Requesters. For the list of supported countries, click here." The login information is correct, since I am able to login to the aws.console.amazon.com page. This is my first time that I use this account with MTurk. Any suggestions on what could be going on here? Thank you very much in advance!lg...

Whenever Turkers submit their assignment, they receive the error message: "Sorry, we couldn't find that page" When opening the web browser console, one Turker said they received this error associated with the POST from pressing submit: "Uncaught SyntaxError: Unexpected token '&' (at submit:26:52)" The HIT generally seems to work properly, even flashing "Loading next HIT" very briefly after submitting, but then posting the above error. Any ideas how to fix this issue? Thanks!lg...

I am trying to create a HIT where the worker presses a key in response to images being displayed. The goal of the HIT is to have many workers perform the same identifications in a random order. For example, worker 1 sees the images in order 1, 2, 3, 4, while worker 2 sees the images in the order 3, 1, 4, 2. Because the response time is essential, the images are displayed one-at-a-time. The worker sees image 1, responds, and then image 1 is replaced with image 2, etc.. I am not clear on what is the best method for doing this and recording the response times. Currently, my plan is to create an text box for each image to be displayed and storing the response time for that image in its corresponding field. So the amount of time to respond to image 1 would be recorded in "TextBox1", the amount of time to respond to image 2 would be recorded in "TextBox2", and so on. Is there a better way to do this? For example, is there a way to retrieve the console log from the worker? That would allow me to write the data to the console and not require additional objects on the page.lg...