UDF throws error when used in query with JOIN

Question

Hello

I created a simple UDF and I am using it in the WHERE clause. It works only if the query has only one table. If I have a JOIN then query throws `GENERIC_INTERNAL_ERROR: Internal error` and no other error details. It is reproduced with **any** user defined function and **any** table. 
Here are queries for example:

Table school:
| Column | Type |
| --- | --- |
| id | int |
| name | string |

Table student:
| Column | Type |
| --- | --- |
| id | int |
| name | string |
| school_id | int |

The following query succeeded:
```
USING EXTERNAL FUNCTION is_true(col1 VARCHAR) RETURNS BOOLEAN LAMBDA 'athena-udf-java11'
select * 
from school 
where is_true(name)
```

This query throws an internal error:
```
USING EXTERNAL FUNCTION is_true(col1 VARCHAR) RETURNS BOOLEAN LAMBDA 'athena-udf-java11'
select * 
from school s 
   JOIN student st ON s.id=st.school_id  
where is_true(s.name)
```

NOTE: using UDF in SELECT clause works successfully, but not in WHERE clause

Answer

Athena team has identified and is currently addressing an issue related to UDFs where dynamic filters are being pushed down the query plan. The current workaround is to modify your query so that dynamic filters are not being used, or to continue using V2. The proposed fix is currently in the testing stage and is expected to be included as part of next release.
 
Athena deploys code on a regular monthly cadence after passing all reliability and performance gates. The deployment then gets rolled out to different regions, and that deployment takes about 2 weeks. However, during the deployment process rollbacks of the release may occur, if any issues are discovered, to ensure customers workloads are not impacted.

UDF throws error when used in query with JOIN

Relevant content