Unable to create Glue Resource Policy

0

Hi,

I'm looking to ditch AWS Lake Formation as a solution to enable cross-account access to my data lake's Glue Data Catalog, in favour of the much simpler Glue Resource Policy solution.

To be clear, I no longer want Lake Formation to have any effect on my permissions model and want to revert to using pure IAM. However, when I try to create a Glue Resource Policy, AWS complains I need to "EnableHybrid" because it thinks I'm still using LakeFormation in addition to the Glue Resource policy. This is the response I get when running aws glue create-resource-policy:

$ aws glue put-resource-policy --cli-input-json file://policy.json  

An error occurred (InvalidInputException) when calling the PutResourcePolicy operation: EnableHybrid must be set to TRUE to allow policy hybrid.  

I've completely deleted every database from my Glue Data Catalog and backed the metadata up locally. This has also removed all data permission grants from AWS Lake Formation. The Lake Formation settings are reverted to the default settings, as shown below:

$ aws lakeformation get-data-lake-settings  
{  
    "DataLakeSettings": {  
        "DataLakeAdmins": \[],  
        "CreateDatabaseDefaultPermissions": \[],  
        "CreateTableDefaultPermissions": \[],  
        "TrustedResourceOwners": \[]  
    }  
}  

$ aws lakeformation list-permissions  
{  
    "PrincipalResourcePermissions": \[]  
}  

I could set EnableHybrid to TRUE, however I don't think this should be necessary, and I worry that all of the error-proneness of Lake Formation will creep in if it's not completely disabled.

Is there a way to completely revert all Lake Formation settings and have my AWS Account behave as if I'd never touched Lake Formation?

Regards,
Toby

已提問 3 年前檢視次數 987 次
1 個回答
0

Sure enough, after putting the resource policy with "EnableHybrid" set to TRUE, the evil AWS Lake Formation still has its say. This is executing in the "borrower" / "external" account:

$ aws athena start-query-execution --query-string 'SELECT * FROM datalake.prod.example' --work-group dev  
{  
    "QueryExecutionId": "REDACTED"  
}  

$ aws athena get-query-execution --query-execution-id "REDACTED"  
{  
    "QueryExecution": {  
        "QueryExecutionId": "REDACTED",  
        "Query": "SELECT * FROM datalake.prod.example",  
        "StatementType": "DML",  
        "ResultConfiguration": {  
            "OutputLocation": "REDACTED"  
        },  
        "QueryExecutionContext": {},  
        "Status": {  
            "State": "FAILED",  
            "StateChangeReason": "Insufficient Lake Formation permission(s) on example (Service: AWSGlue; Status Code: 400; Error Code: AccessDeniedException; Request ID: REDACTED; Proxy: null)",  
            "SubmissionDateTime": "2021-08-22T12:02:37.211000+10:00",  
            "CompletionDateTime": "2021-08-22T12:02:48.740000+10:00"  
        },  
        "Statistics": {  
            "EngineExecutionTimeInMillis": 10787,  
            "DataScannedInBytes": 0,  
            "TotalExecutionTimeInMillis": 11529,  
            "QueryQueueTimeInMillis": 641,  
            "ServiceProcessingTimeInMillis": 101  
        },  
        "WorkGroup": "dev",  
        "EngineVersion": {  
            "SelectedEngineVersion": "Athena engine version 2",  
            "EffectiveEngineVersion": "Athena engine version 2"  
        }  
    }  
}
已回答 3 年前

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南