Skip to content
By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post
About re:Post

Using the new S3 Metadata and S3 Tables features - catalog does not exist

0

Hello,

I'm trying to use the S3 Metadata feature released this week. To do that, I've created an S3 Table Bucket in us-east-1, and configured my bucket to push its metadata to said table. I've also made sure that Integration with AWS analytics services shows Enabled status.

I am now trying to query my new S3 Table to see if there data has been populated correctly. In S3, I can see the table bucket exists; in the source bucket, it shows Status: Active as well. Copying the data from Destination table bucket, Namespace and Metadata table name here, I tried running the following Athena query:

SELECT * FROM "s3tablescatalog/<dentination-bucket>"."<namespace>"."<table-name>"

However, I receive the error

CATALOG_NOT_FOUND: line 1:15: Catalog 's3tablescatalog/<dentination-bucket>' does not exist

I've verified that I'm running this query in us-east-1, the same region as the source bucket and destination table bucket.

What am I missing?

  • Andreea
    a year ago

    Have u tried going into Lake formation > Catalogs > select your s3 tables catalog > Permissions / Grant > Principals and select the IAM user of the task

    In LF-Tags or catalog resources select Named Data Catalog resources, in Catalogs choose your s3tables catalog and in Permissions give super admin for testing (you can go more granular afterward)

Topics
AnalyticsStorage
Tags
Amazon AthenaAmazon Simple Storage Service
Language
English

asked 2 years ago2.9K views

1 Answer
-1

Thank you for providing details about your issue with the new Amazon S3 Metadata feature. It appears you're encountering a challenge when trying to query your S3 Table using Amazon Athena. Let me help clarify a few points and suggest a potential solution.

First, it's important to note that S3 Metadata is currently in preview, and its integration with AWS Glue Data Catalog is also in preview. This integration is what allows you to query S3 Metadata tables using services like Amazon Athena.

The error message you're receiving suggests that Athena is unable to find the catalog you're referencing. This could be because the integration between S3 Tables and AWS Glue Data Catalog may not be fully established yet, despite the "Enabled" status you're seeing.

Here are a few things you can try:

  1. Instead of using "s3tablescatalog/<destination-bucket>" in your query, try using just "AwsDataCatalog" as the catalog name. The correct query format should look like this:

    SELECT * FROM "AwsDataCatalog"."<namespace>"."<table-name>"

  2. Verify that the S3 Metadata table has been properly registered in the AWS Glue Data Catalog. You might need to check the AWS Glue console to ensure the table is visible there.

  3. Make sure you have the necessary permissions to access both the S3 bucket and the AWS Glue Data Catalog.

  4. Double-check that enough time has passed for the metadata to be populated in the S3 Table. While S3 Metadata updates tables within minutes of changes, there might be a slight delay before the data is queryable.

  5. If these steps don't resolve the issue, you may need to contact AWS Support, as there could be a configuration issue related to the preview status of this feature.

Remember, S3 Metadata and its integration with AWS analytics services are new features still in preview, so some aspects of their functionality may be subject to change or improvement. If you continue to experience issues, don't hesitate to reach out to AWS Support for further assistance.
Sources
Announcing Amazon S3 Metadata (Preview) – Easiest and fastest way to manage your metadata - AWS
Data Discovery Accelerator - Amazon S3 Metadata - AWS

answered 2 years ago

  • User-7942380
    2 years ago

    I've tried the Agent's suggestions.

    1. Changing to AwsDataCatalog just causes a SCHEMA_NOT_FOUND error as the schema aws_s3_metadata only exists in the s3tablescatalog catalog - and not in the AwsDataCatalog.
    2. I've tried manually registering the table via the CLI and it says it already exists. In the console, the table is not showing up - but I cannot see the s3tablescatalog schema
    3. I have checked that I have the required permissions
    4. I've waited a few hours and the problem persists

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Relevant content