BatchWrite limits

0

Related a bit to forum https://forums.aws.amazon.com/post!reply.jspa?messageID=821050, but I don't think I should completely hijack that topic.

I'm performing the following batch_write operation using Python:

{
    'DirectoryArn': '<snip>', 
    'Operations': [
        {'CreateObject': {'SchemaFacet': [{'SchemaArn': '<snip>', 'FacetName': 'Organization'}], 'ObjectAttributeList': [{'Key': {'SchemaArn': '<snip>', 'FacetName': 'Organization', 'Name': 'display_name'}, 'Value': {'StringValue': 'Object #1'}}], 'ParentReference': {'Selector': '/'}, 'LinkName': 'object', 'BatchReferenceName': '5bd5059b-b793-4ebe-b45b-b0a136a4ac8d'}},
        {'CreateObject': {'SchemaFacet': [{'SchemaArn': '<snip>', 'FacetName': 'Meta'}], 'ObjectAttributeList': [{'Key': {'SchemaArn': '<snip>', 'FacetName': 'Meta', 'Name': 'display_name'}, 'Value': {'StringValue': 'Child #1'}}], 'ParentReference': {'Selector': '#5bd5059b-b793-4ebe-b45b-b0a136a4ac8d'}, 'LinkName': 'organization', 'BatchReferenceName': '01e66be7-d47f-40ed-b7d0-faa638db9ced'}},
        {'CreateObject': {'SchemaFacet': [{'SchemaArn': '<snip>', 'FacetName': 'Meta'}], 'ObjectAttributeList': [{'Key': {'SchemaArn': '<snip>', 'FacetName': 'Meta', 'Name': 'display_name'}, 'Value': {'StringValue': 'Child #2'}}], 'ParentReference': {'Selector': '#5bd5059b-b793-4ebe-b45b-b0a136a4ac8d'}, 'LinkName': 'index', 'BatchReferenceName': 'f8b672b9-fef5-40c8-8c55-94152d4db745'}},
        {'CreateObject': {'SchemaFacet': [{'SchemaArn': '<snip>', 'FacetName': 'INDEX'}], 'ObjectAttributeList': [], 'ParentReference': {'Selector': '#f8b672b9-fef5-40c8-8c55-94152d4db745'}, 'LinkName': 'index1', 'BatchReferenceName': '71b75012-516b-4cb6-8354-976ad1cc4589'}},
        {'CreateObject': {'SchemaFacet': [{'SchemaArn': '<snip>', 'FacetName': 'INDEX'}], 'ObjectAttributeList': [], 'ParentReference': {'Selector': '#f8b672b9-fef5-40c8-8c55-94152d4db745'}, 'LinkName': 'index2', 'BatchReferenceName': 'c6824a61-1338-4cd5-8d3a-553b67e50a18'}},
        {'CreateObject': {'SchemaFacet': [{'SchemaArn': '<snip>', 'FacetName': 'INDEX'}], 'ObjectAttributeList': [], 'ParentReference': {'Selector': '#f8b672b9-fef5-40c8-8c55-94152d4db745'}, 'LinkName': 'index3', 'BatchReferenceName': 'c72c0bab-25f1-4a83-a9c5-569d34d1a90d'}},
        {'CreateObject': {'SchemaFacet': [{'SchemaArn': '<snip>', 'FacetName': 'INDEX'}], 'ObjectAttributeList': [], 'ParentReference': {'Selector': '#f8b672b9-fef5-40c8-8c55-94152d4db745'}, 'LinkName': 'index4', 'BatchReferenceName': '8b4368e2-2d29-4666-8eb0-4e93105f1efb'}},
        {'CreateObject': {'SchemaFacet': [{'SchemaArn': '<snip>', 'FacetName': 'INDEX'}], 'ObjectAttributeList': [], 'ParentReference': {'Selector': '#f8b672b9-fef5-40c8-8c55-94152d4db745'}, 'LinkName': 'index5', 'BatchReferenceName': 'e5ba15a8-4cdd-43a7-873e-0afd8c0e6870'}},
        {'CreateObject': {'SchemaFacet': [{'SchemaArn': '<snip>', 'FacetName': 'INDEX'}], 'ObjectAttributeList': [], 'ParentReference': {'Selector': '#f8b672b9-fef5-40c8-8c55-94152d4db745'}, 'LinkName': 'index6', 'BatchReferenceName': 'afba33be-d2a2-461f-8742-ec0cb805b0dc'}},
        {'CreateObject': {'SchemaFacet': [{'SchemaArn': '<snip>', 'FacetName': 'INDEX'}], 'ObjectAttributeList': [], 'ParentReference': {'Selector': '#f8b672b9-fef5-40c8-8c55-94152d4db745'}, 'LinkName': 'index7', 'BatchReferenceName': 'c4b70cd5-1a39-45ba-a5ee-efb0fbc424e7'}},
        {'CreateObject': {'SchemaFacet': [{'SchemaArn': '<snip>', 'FacetName': 'INDEX'}], 'ObjectAttributeList': [], 'ParentReference': {'Selector': '#f8b672b9-fef5-40c8-8c55-94152d4db745'}, 'LinkName': 'index8', 'BatchReferenceName': '1ac0290c-339f-4b50-9080-d1c5c8d4159a'}}
    ]
}

which fails with the error:

An error occurred (LimitExceededException) when calling the BatchWrite operation: Limits exceeded on the request. Write objects, max: 20, actual: 22. Read objects, max: 200, actual: 10. Write attributes, max: 1000, actual: 4. Read attributes, max: 1000, actual: 0.

I'm creating 11 objects. The selector paths would look like:

/object
/object/organization
/object/index
/object/index/index1
/object/index/index2
/object/index/index3
/object/index/index4
/object/index/index5
/object/index/index6
/object/index/index7
/object/index/index8

Although I can understand where the 22 Writes come from, I wonder if this is a fair assumption as the object with "/object/index" is for example counted 9 times (1 create, 8 updates). I'm using the BatchReference so during the counting it can be taken into account that I intend to write the same object multiple times, and as it is documented that the batch_write is going to be a single transaction, I would expect my "Write object count" to be 12 (11 objects created + root node update.

This comment can also be extended a bit to the Read Objects count of 10, but I'll that one out of the discussion. :)

idoorn
asked 5 years ago178 views
2 Answers
0

Apologies, the wording of the error message could be improved. I'll create an item in our internal bug tracker.

You are correct that there are 11 Objects created. When the error message says "Write objects, max: 20, actual: 22", it is using the "little o" definition of object (anything in your directory) rather than the "big O" definition of Object (a node in the directory's graph). This can certainly cause confusion. In your case, the additional writes are coming from the creation of the links from "/object/index" to each child. Both Links and Objects count towards this write limit.

-Matt

AWS
answered 5 years ago
0

Thanks for the clarification.

idoorn
answered 5 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions