Gremlin Query to count Edges - MemoryLimitExceededException AWS Neptune

0

Gremlin Query on Neptune to count no of edges for vertexes with certain label results in MemoryLimitExceededException. Error only occurs a few times i.e. if i run query 10 times, succeeds twice.

Neptune writer is setup with db.r5.xlarge.

Query

g.V().hasLabel('MyVertex').has('myLabel',1234).outE().hasLabel('MY_EDGE_LABEL').count().next()
There are over 3 million vertices with label MyVertex and 1 million edges matching MY_EDGE_LABEL edge label.

Error:

{
"code": "MemoryLimitExceededException",
"requestId": "123456-abcd-1234-1234-11aa221122ab",
"detailedMessage": "Query cannot be completed due to memory limitations."
}

My Questions:
Is there any setting like memory limit for a query on Neptune that can be increased?
Is there anything in the query that can be adjusted to get same result but consumes less memory.

Edited by: BaluVV on Mar 7, 2021 8:04 PM

Edited by: BaluVV on Mar 7, 2021 8:07 PM

Edited by: BaluVV on Mar 7, 2021 8:07 PM

BaluVV
질문됨 3년 전1340회 조회
3개 답변
0

Hi,
is it possible to reach us via a ticket (TT). That way we have access to your instance and we can investigate deeper. For example it would be interesting to know what would be the size of the result approximately.

As an alternative for now I wonder if you can split the query in multiple steps, each of them retrieving a subset of the data.

Gremlin also has support for query profiling but this is only useful when query completes: https://docs.aws.amazon.com/neptune/latest/userguide/gremlin-profile-api.html

--Gabriel

답변함 3년 전
0

Adding limit(1) fixed it, not sure how though.

g.V().hasLabel('MyVertex').has('myLabel',1234).outE().hasLabel('MY_EDGE_LABEL').count().limit(1).next()

BaluVV
답변함 3년 전
0

Hi BaluVV,

Neptune engine for Gremlin processes solutions in chunks and specifying a limit(1) has the behavior of hinting the engine to process one solution at a time at each query step (node) instead of processing a chunk at a time. Setting a lower chunk size has a proportional effect on latency. You can observe the same indicated using "chunkSize" attribute in the query profile or explain output. If you need more assistance, please reach out through a support ticket.

Thanks.

AWS
답변함 3년 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인