UDF throws error when used in query with JOIN

0

Hello

I created a simple UDF and I am using it in the WHERE clause. It works only if the query has only one table. If I have a JOIN then query throws GENERIC_INTERNAL_ERROR: Internal error and no other error details. It is reproduced with any user defined function and any table. Here are queries for example:

Table school:

ColumnType
idint
namestring

Table student:

ColumnType
idint
namestring
school_idint

The following query succeeded:

USING EXTERNAL FUNCTION is_true(col1 VARCHAR) RETURNS BOOLEAN LAMBDA 'athena-udf-java11'
select * 
from school 
where is_true(name)

This query throws an internal error:

USING EXTERNAL FUNCTION is_true(col1 VARCHAR) RETURNS BOOLEAN LAMBDA 'athena-udf-java11'
select * 
from school s 
   JOIN student st ON s.id=st.school_id  
where is_true(s.name)

NOTE: using UDF in SELECT clause works successfully, but not in WHERE clause

질문됨 8달 전225회 조회
1개 답변
0

Athena team has identified and is currently addressing an issue related to UDFs where dynamic filters are being pushed down the query plan. The current workaround is to modify your query so that dynamic filters are not being used, or to continue using V2. The proposed fix is currently in the testing stage and is expected to be included as part of next release.

Athena deploys code on a regular monthly cadence after passing all reliability and performance gates. The deployment then gets rolled out to different regions, and that deployment takes about 2 weeks. However, during the deployment process rollbacks of the release may occur, if any issues are discovered, to ensure customers workloads are not impacted.

AWS
답변함 8달 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠