Redshift Spectrum giving IERR_READ_ERROR (Access Denied) error

0

I'm trying to use Redshift Spectrum to query gz JSON data in S3. I've used a Glue crawler to create the database and table schema which all appears to be correct. I've created the external schema using

create external schema spectrum from data catalog ...

which populates my SQL IDE with the schema from Glue. But when I go to query data from this table, I get the following error:

S3 Query Exception (Fetch). Task failed due to an internal error. 
Error: HTTP response error code: 403 Message: AccessDenied Access Denied
x-amz-request-id: xxx
x-amz-id-2: xxx
Ion/JSON Stream error while opening buffer: IERR_READ_ERROR
 (s3://<bucket>/.../<file...>...

The S3 data is in a separate AWS account from the Redshift cluster, but I have the S3 bucket policy set up to allow the spectrum-role access based on this link: https://docs.aws.amazon.com/redshift/latest/dg/c-spectrum-iam-policies.html

I've given the spectrum-role full get and list access:

{
    "Sid": "Redshift Spectrum permissions",
    "Effect": "Allow",
    "Principal": {
        "AWS": "arn:aws:iam::<account>:role/spectrum-role"
    },
    "Action": [
        "s3:Get*",
        "s3:List*"
    ],
    "Resource": [
        "arn:aws:s3:::<bucket>",
        "arn:aws:s3:::<bucket>/*"
    ]
}

This thread https://forums.aws.amazon.com/thread.jspa?threadID=280879 suggests that IERR_READ_ERROR is always malformed gz JSON, but seeing "AccessDenied" in the error message makes me question that.

Can anyone help me diagnose what the problem is?

kwhite
preguntada hace 4 años1006 visualizaciones
1 Respuesta
0

The Redshift AWS account did not have permissions for spectrum-role to read from the cross-account AWS S3 bucket (the glue role did, but not the spectrum-role). Adding Get** and List** permissions to the spectrum-role allowed the query to complete.

kwhite
respondido hace 4 años

No has iniciado sesión. Iniciar sesión para publicar una respuesta.

Una buena respuesta responde claramente a la pregunta, proporciona comentarios constructivos y fomenta el crecimiento profesional en la persona que hace la pregunta.

Pautas para responder preguntas