Is it possible to use S3 select to identify schema of data in a S3 file

Yes, Amazon S3 Select does not support automatic schema inference, meaning it cannot directly identify or return the list of first-level fields (keys) from JSON records stored in an S3 object. As stated in the AWS documentation, "S3 Select supports querying data that is structured or semi-structured (like CSV or JSON), but does not support schema inference." This means you must already know the field names and structure of your data to write SQL queries against it.

To work around this limitation, you can use alternative tools such as AWS Glue , which can automatically infer schemas from JSON files and generate metadata tables for use with Amazon Athena or other query services. Another option is using a custom AWS Lambda function or a local script (e.g., Python with Pandas or json-schema-infer) to read sample data from the file and programmatically extract all unique top-level keys across the JSON records.

Reference: AWS S3 Select Documentation - JSON Support