We have a file that we used the default XML crawler to crawl the data for, and it correctly created a table and schema for the data (relevant column shown):
However when creating a job in the Visual ETL tool it changes the datatype from bigint to choice in the data preview session, and shows the data as mostly longs, with a couple of int values:
Attempting to do a change schema node doesn't allow me to change the choice value:
Trying to do any data transform (such as flatten) results in an error:
Things I've tried to do to correct the issue:
- Change the schema type in the table from bigint to int.
- Use the
resolveChoice
function
- Restarting data preview sessions
- Recreating the job / data catalog source node
- Recrawl the data
Also, if i don't use the data preview session, but just run the job, it runs successfully and outputs the data as expected, so it appears to just be a problem with the data preview (however this renders data preview useless).
Is there any way to resolve this issue? We've had it show up not just in this set of data, but multiple sets of data over the last few days, and we hadn't seen it before last friday (Jan 19, 2024).