跳至内容

Athena-DocDb connector - Getting "Unable to make field long java.nio.Buffer.address accessible" error

0

I am new to Athena, and referred this page to create Document DB connector.
Background: I had to create the lambda manually by forking the code as the assumed role doesn't have permissions to access the Serverless Application Repository; defined only one environment variable 'default_docdb' that contains the db url. The lambda uses Java 17.

When I access any collection from this database, I see this error from the lambda logs. I found a stackoverlfow question discussing the same issue. Did I miss any configuration steps while deploying the lambda manually?

java.lang.RuntimeException: Failed to initialize MemoryUtil. Was Java started with `--add-opens=java.base/java.nio=ALL-UNNAMED`? (See https://arrow.apache.org/docs/java/install.html)|
at org.apache.arrow.memory.util.MemoryUtil.<clinit>(MemoryUtil.java:143)|
at org.apache.arrow.memory.ArrowBuf.setZero(ArrowBuf.java:1175) |
at org.apache.arrow.vector.BaseFixedWidthVector.reAlloc(BaseFixedWidthVector.java:445)|
at org.apache.arrow.vector.BaseFixedWidthVector.handleSafe(BaseFixedWidthVector.java:850) |
at org.apache.arrow.vector.IntVector.setSafe(IntVector.java:203)|
at com.amazonaws.athena.connector.lambda.data.BlockUtils.setValue(BlockUtils.java:309)|
at com.amazonaws.athena.connector.lambda.data.BlockUtils.newBlock(BlockUtils.java:151)|
at com.amazonaws.athena.connector.lambda.data.BlockUtils.newBlock(BlockUtils.java:130)|
at com.amazonaws.athena.connector.lambda.handlers.MetadataHandler.doGetTableLayout(MetadataHandler.java:406)|
at com.amazonaws.athena.connector.lambda.handlers.MetadataHandler.doHandleRequest(MetadataHandler.java:283) |
at com.amazonaws.athena.connector.lambda.handlers.CompositeHandler.handleRequest(CompositeHandler.java:154) |
at com.amazonaws.athena.connector.lambda.handlers.CompositeHandler.handleRequest(CompositeHandler.java:122) |
at com.amazonaws.services.lambda.runtime.api.client.EventHandlerLoader$2.call(EventHandlerLoader.java:907)|
at com.amazonaws.services.lambda.runtime.api.client.AWSLambda.startRuntime(AWSLambda.java:239)|
at com.amazonaws.services.lambda.runtime.api.client.AWSLambda.startRuntime(AWSLambda.java:191)|
at com.amazonaws.services.lambda.runtime.api.client.AWSLambda.main(AWSLambda.java:181)|
Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make field long java.nio.Buffer.address accessible: module java.base does not "opens java.nio" to unnamed module @35bbe5e8|
at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(Unknown Source) |
at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(Unknown Source) |
at java.base/java.lang.reflect.Field.checkCanSetAccessible(Unknown Source)|
at java.base/java.lang.reflect.Field.setAccessible(Unknown Source)|
at org.apache.arrow.memory.util.MemoryUtil.<clinit>(MemoryUtil.java:88) |
... 15 more |```
已提问 1 年前636 查看次数
1 回答
1
已接受的回答

Through the document - https://arrow.apache.org/docs/java/install.html. You can see for Jave 9 or later, some JDK internals must be exposed by adding --add-opens=java.base/java.nio=org.apache.arrow.memory.core,ALL-UNNAMED to the Java.

Explain:

“add-opens” is a command-line option in Java used to resolve module system access restrictions. Since the introduction of the module system in Java 9, certain internal APIs (such as java.nio) are hidden and restricted by default. If a library or program (such as Apache Arrow) needs access to these internal APIs, it will encounter access restrictions, resulting in runtime errors like module java.base does not "opens java.nio" to unnamed module. The command -add-opens=java.base/java.nio=org.apache.arrow.memory.core,ALL-UNNAMED allows a Java program at runtime to grant the org.apache.arrow.memory.core module access to the java.nio package within the java.base module, thereby lifting the access restriction.

AWS
已回答 1 年前
专家
已审核 1 年前
专家
已审核 1 年前
  • Thank you - after adding this option, the error disappeared, but I cannot query any collections. I tried a select * (getting "SELECT * not allowed from relation that has no columns") as well as a select column (getting "Column 'createdat' cannot be resolved or requester is not authorized to access requested resources".

您未登录。 登录 发布回答。

一个好的回答可以清楚地解答问题和提供建设性反馈,并能促进提问者的职业发展。