跳至內容

Athena-DocDb connector - Getting "Unable to make field long java.nio.Buffer.address accessible" error

0

I am new to Athena, and referred this page to create Document DB connector.
Background: I had to create the lambda manually by forking the code as the assumed role doesn't have permissions to access the Serverless Application Repository; defined only one environment variable 'default_docdb' that contains the db url. The lambda uses Java 17.

When I access any collection from this database, I see this error from the lambda logs. I found a stackoverlfow question discussing the same issue. Did I miss any configuration steps while deploying the lambda manually?

java.lang.RuntimeException: Failed to initialize MemoryUtil. Was Java started with `--add-opens=java.base/java.nio=ALL-UNNAMED`? (See https://arrow.apache.org/docs/java/install.html)|
at org.apache.arrow.memory.util.MemoryUtil.<clinit>(MemoryUtil.java:143)|
at org.apache.arrow.memory.ArrowBuf.setZero(ArrowBuf.java:1175) |
at org.apache.arrow.vector.BaseFixedWidthVector.reAlloc(BaseFixedWidthVector.java:445)|
at org.apache.arrow.vector.BaseFixedWidthVector.handleSafe(BaseFixedWidthVector.java:850) |
at org.apache.arrow.vector.IntVector.setSafe(IntVector.java:203)|
at com.amazonaws.athena.connector.lambda.data.BlockUtils.setValue(BlockUtils.java:309)|
at com.amazonaws.athena.connector.lambda.data.BlockUtils.newBlock(BlockUtils.java:151)|
at com.amazonaws.athena.connector.lambda.data.BlockUtils.newBlock(BlockUtils.java:130)|
at com.amazonaws.athena.connector.lambda.handlers.MetadataHandler.doGetTableLayout(MetadataHandler.java:406)|
at com.amazonaws.athena.connector.lambda.handlers.MetadataHandler.doHandleRequest(MetadataHandler.java:283) |
at com.amazonaws.athena.connector.lambda.handlers.CompositeHandler.handleRequest(CompositeHandler.java:154) |
at com.amazonaws.athena.connector.lambda.handlers.CompositeHandler.handleRequest(CompositeHandler.java:122) |
at com.amazonaws.services.lambda.runtime.api.client.EventHandlerLoader$2.call(EventHandlerLoader.java:907)|
at com.amazonaws.services.lambda.runtime.api.client.AWSLambda.startRuntime(AWSLambda.java:239)|
at com.amazonaws.services.lambda.runtime.api.client.AWSLambda.startRuntime(AWSLambda.java:191)|
at com.amazonaws.services.lambda.runtime.api.client.AWSLambda.main(AWSLambda.java:181)|
Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make field long java.nio.Buffer.address accessible: module java.base does not "opens java.nio" to unnamed module @35bbe5e8|
at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(Unknown Source) |
at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(Unknown Source) |
at java.base/java.lang.reflect.Field.checkCanSetAccessible(Unknown Source)|
at java.base/java.lang.reflect.Field.setAccessible(Unknown Source)|
at org.apache.arrow.memory.util.MemoryUtil.<clinit>(MemoryUtil.java:88) |
... 15 more |```
已提問 1 年前檢視次數 628 次
1 個回答
1
已接受的答案

Through the document - https://arrow.apache.org/docs/java/install.html. You can see for Jave 9 or later, some JDK internals must be exposed by adding --add-opens=java.base/java.nio=org.apache.arrow.memory.core,ALL-UNNAMED to the Java.

Explain:

“add-opens” is a command-line option in Java used to resolve module system access restrictions. Since the introduction of the module system in Java 9, certain internal APIs (such as java.nio) are hidden and restricted by default. If a library or program (such as Apache Arrow) needs access to these internal APIs, it will encounter access restrictions, resulting in runtime errors like module java.base does not "opens java.nio" to unnamed module. The command -add-opens=java.base/java.nio=org.apache.arrow.memory.core,ALL-UNNAMED allows a Java program at runtime to grant the org.apache.arrow.memory.core module access to the java.nio package within the java.base module, thereby lifting the access restriction.

AWS
已回答 1 年前
專家
已審閱 1 年前
專家
已審閱 1 年前
  • Thank you - after adding this option, the error disappeared, but I cannot query any collections. I tried a select * (getting "SELECT * not allowed from relation that has no columns") as well as a select column (getting "Column 'createdat' cannot be resolved or requester is not authorized to access requested resources".

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。