Redshift Spectrum giving IERR_READ_ERROR (Access Denied) error

0

I'm trying to use Redshift Spectrum to query gz JSON data in S3. I've used a Glue crawler to create the database and table schema which all appears to be correct. I've created the external schema using

create external schema spectrum from data catalog ...

which populates my SQL IDE with the schema from Glue. But when I go to query data from this table, I get the following error:

S3 Query Exception (Fetch). Task failed due to an internal error. 
Error: HTTP response error code: 403 Message: AccessDenied Access Denied
x-amz-request-id: xxx
x-amz-id-2: xxx
Ion/JSON Stream error while opening buffer: IERR_READ_ERROR
 (s3://<bucket>/.../<file...>...

The S3 data is in a separate AWS account from the Redshift cluster, but I have the S3 bucket policy set up to allow the spectrum-role access based on this link: https://docs.aws.amazon.com/redshift/latest/dg/c-spectrum-iam-policies.html

I've given the spectrum-role full get and list access:

{
    "Sid": "Redshift Spectrum permissions",
    "Effect": "Allow",
    "Principal": {
        "AWS": "arn:aws:iam::<account>:role/spectrum-role"
    },
    "Action": [
        "s3:Get*",
        "s3:List*"
    ],
    "Resource": [
        "arn:aws:s3:::<bucket>",
        "arn:aws:s3:::<bucket>/*"
    ]
}

This thread https://forums.aws.amazon.com/thread.jspa?threadID=280879 suggests that IERR_READ_ERROR is always malformed gz JSON, but seeing "AccessDenied" in the error message makes me question that.

Can anyone help me diagnose what the problem is?

kwhite
posta 4 anni fa1006 visualizzazioni
1 Risposta
0

The Redshift AWS account did not have permissions for spectrum-role to read from the cross-account AWS S3 bucket (the glue role did, but not the spectrum-role). Adding Get** and List** permissions to the spectrum-role allowed the query to complete.

kwhite
con risposta 4 anni fa

Accesso non effettuato. Accedi per postare una risposta.

Una buona risposta soddisfa chiaramente la domanda, fornisce un feedback costruttivo e incoraggia la crescita professionale del richiedente.

Linee guida per rispondere alle domande