2 Respuestas
- Más nuevo
- Más votos
- Más comentarios
0
Under options there are two parms. One is retention period and the other is manifest file. The retention period is 7 days by default and manifest file path might show what files are deleted successfully and which ones failed. Hope that helps. https://docs.aws.amazon.com/glue/latest/dg/aws-glue-api-crawler-pyspark-extensions-glue-context.html
respondido hace 2 años
Thank you for your help, it gave me a hint of should I look. Unfortunatelly, I wasn't able to fix it yet. It seems like none of the files appears in the manifest folder, just some partitions that I'm not sure of what they represent.
0
Hopefully you worked this out at the time, but I just also hit this issue. It appears you have to over-right the retention period set to 0 to delete immediately.
glueContext.purge_s3_path("s3://my_bucket/path_to_purge/", options={"retentionPeriod": 0}, transformation_ctx="")
respondido hace un año
Contenido relevante
- OFICIAL DE AWSActualizada hace 2 años
- OFICIAL DE AWSActualizada hace un año
- OFICIAL DE AWSActualizada hace 8 meses
The problem was due to lack of permissions for deleting objets in the role created for the jobs. Once revised the roles permissions, the query worked correctly
Sorry to revive this old thread. Could you help me know how did you setup the permission?
Thanks a lot, Haoran