Adding to the result.json

Hi AussieV123, the results of your run are being stored on S3 under <your-bucket>/<your-prefix>/<task-id>/results.json. While is is not possible at the moment to write meta data directly in the results.json file, you can add meta data manually in the result s3 folder (<your-bucket>/<your-prefix>/<task-id>).

Would that address your use case?

Edited by: EricMK-AWS on Sep 9, 2020 9:15 AM

Yes, I could add information to the bucket but if I have multiple requests coming in from a Lambda function for example is there any way of telling them apart other than order?

If for instance I had the task ID as in (<your-bucket>/<your-prefix>/<task-id>) I could store this into DynamoDB on or after the task launched. Then when the new result file showed up in S3 it could trigger a Lambda function which would read the folder name, retrieve the result.json and lookup the meta information that would have been stored in the result.json.

It could work if I knew the task-id at time of submission. Can I get the task-id on submission?

I could also use <your-prefix> in a similar manner and generate a UUID or a similar id/hash to lock it to an entry in DynamoDB.

Seems like a lot of extra steps for a little meta data.

Hi AussieV123, thanks a lot for the explanation and detail on your use case. Yes, the task ID is created when you submit a task with a .run() command. You can retrieve it like this:

task = device.run(my_circuit, s3_folder= (your_bucket, your_prefix), ...)
task_id = task.id.split('/')[1] # technically the task ID is a full ARN but we are only interested in the random string at the end

You can then use boto3 to put your meta data in the same s3 folder the results will live in (your_bucket/your_prefix/task_id).