Unable to create Glue Resource Policy

0

Hi,

I'm looking to ditch AWS Lake Formation as a solution to enable cross-account access to my data lake's Glue Data Catalog, in favour of the much simpler Glue Resource Policy solution.

To be clear, I no longer want Lake Formation to have any effect on my permissions model and want to revert to using pure IAM. However, when I try to create a Glue Resource Policy, AWS complains I need to "EnableHybrid" because it thinks I'm still using LakeFormation in addition to the Glue Resource policy. This is the response I get when running aws glue create-resource-policy:

$ aws glue put-resource-policy --cli-input-json file://policy.json  

An error occurred (InvalidInputException) when calling the PutResourcePolicy operation: EnableHybrid must be set to TRUE to allow policy hybrid.  

I've completely deleted every database from my Glue Data Catalog and backed the metadata up locally. This has also removed all data permission grants from AWS Lake Formation. The Lake Formation settings are reverted to the default settings, as shown below:

$ aws lakeformation get-data-lake-settings  
{  
    "DataLakeSettings": {  
        "DataLakeAdmins": \[],  
        "CreateDatabaseDefaultPermissions": \[],  
        "CreateTableDefaultPermissions": \[],  
        "TrustedResourceOwners": \[]  
    }  
}  

$ aws lakeformation list-permissions  
{  
    "PrincipalResourcePermissions": \[]  
}  

I could set EnableHybrid to TRUE, however I don't think this should be necessary, and I worry that all of the error-proneness of Lake Formation will creep in if it's not completely disabled.

Is there a way to completely revert all Lake Formation settings and have my AWS Account behave as if I'd never touched Lake Formation?

Regards,
Toby

已提问 3 年前987 查看次数
1 回答
0

Sure enough, after putting the resource policy with "EnableHybrid" set to TRUE, the evil AWS Lake Formation still has its say. This is executing in the "borrower" / "external" account:

$ aws athena start-query-execution --query-string 'SELECT * FROM datalake.prod.example' --work-group dev  
{  
    "QueryExecutionId": "REDACTED"  
}  

$ aws athena get-query-execution --query-execution-id "REDACTED"  
{  
    "QueryExecution": {  
        "QueryExecutionId": "REDACTED",  
        "Query": "SELECT * FROM datalake.prod.example",  
        "StatementType": "DML",  
        "ResultConfiguration": {  
            "OutputLocation": "REDACTED"  
        },  
        "QueryExecutionContext": {},  
        "Status": {  
            "State": "FAILED",  
            "StateChangeReason": "Insufficient Lake Formation permission(s) on example (Service: AWSGlue; Status Code: 400; Error Code: AccessDeniedException; Request ID: REDACTED; Proxy: null)",  
            "SubmissionDateTime": "2021-08-22T12:02:37.211000+10:00",  
            "CompletionDateTime": "2021-08-22T12:02:48.740000+10:00"  
        },  
        "Statistics": {  
            "EngineExecutionTimeInMillis": 10787,  
            "DataScannedInBytes": 0,  
            "TotalExecutionTimeInMillis": 11529,  
            "QueryQueueTimeInMillis": 641,  
            "ServiceProcessingTimeInMillis": 101  
        },  
        "WorkGroup": "dev",  
        "EngineVersion": {  
            "SelectedEngineVersion": "Athena engine version 2",  
            "EffectiveEngineVersion": "Athena engine version 2"  
        }  
    }  
}
已回答 3 年前

您未登录。 登录 发布回答。

一个好的回答可以清楚地解答问题和提供建设性反馈,并能促进提问者的职业发展。

回答问题的准则