Unable to create Glue Resource Policy

0

Hi,

I'm looking to ditch AWS Lake Formation as a solution to enable cross-account access to my data lake's Glue Data Catalog, in favour of the much simpler Glue Resource Policy solution.

To be clear, I no longer want Lake Formation to have any effect on my permissions model and want to revert to using pure IAM. However, when I try to create a Glue Resource Policy, AWS complains I need to "EnableHybrid" because it thinks I'm still using LakeFormation in addition to the Glue Resource policy. This is the response I get when running aws glue create-resource-policy:

$ aws glue put-resource-policy --cli-input-json file://policy.json  

An error occurred (InvalidInputException) when calling the PutResourcePolicy operation: EnableHybrid must be set to TRUE to allow policy hybrid.  

I've completely deleted every database from my Glue Data Catalog and backed the metadata up locally. This has also removed all data permission grants from AWS Lake Formation. The Lake Formation settings are reverted to the default settings, as shown below:

$ aws lakeformation get-data-lake-settings  
{  
    "DataLakeSettings": {  
        "DataLakeAdmins": \[],  
        "CreateDatabaseDefaultPermissions": \[],  
        "CreateTableDefaultPermissions": \[],  
        "TrustedResourceOwners": \[]  
    }  
}  

$ aws lakeformation list-permissions  
{  
    "PrincipalResourcePermissions": \[]  
}  

I could set EnableHybrid to TRUE, however I don't think this should be necessary, and I worry that all of the error-proneness of Lake Formation will creep in if it's not completely disabled.

Is there a way to completely revert all Lake Formation settings and have my AWS Account behave as if I'd never touched Lake Formation?

Regards,
Toby

1回答
0

Sure enough, after putting the resource policy with "EnableHybrid" set to TRUE, the evil AWS Lake Formation still has its say. This is executing in the "borrower" / "external" account:

$ aws athena start-query-execution --query-string 'SELECT * FROM datalake.prod.example' --work-group dev  
{  
    "QueryExecutionId": "REDACTED"  
}  

$ aws athena get-query-execution --query-execution-id "REDACTED"  
{  
    "QueryExecution": {  
        "QueryExecutionId": "REDACTED",  
        "Query": "SELECT * FROM datalake.prod.example",  
        "StatementType": "DML",  
        "ResultConfiguration": {  
            "OutputLocation": "REDACTED"  
        },  
        "QueryExecutionContext": {},  
        "Status": {  
            "State": "FAILED",  
            "StateChangeReason": "Insufficient Lake Formation permission(s) on example (Service: AWSGlue; Status Code: 400; Error Code: AccessDeniedException; Request ID: REDACTED; Proxy: null)",  
            "SubmissionDateTime": "2021-08-22T12:02:37.211000+10:00",  
            "CompletionDateTime": "2021-08-22T12:02:48.740000+10:00"  
        },  
        "Statistics": {  
            "EngineExecutionTimeInMillis": 10787,  
            "DataScannedInBytes": 0,  
            "TotalExecutionTimeInMillis": 11529,  
            "QueryQueueTimeInMillis": 641,  
            "ServiceProcessingTimeInMillis": 101  
        },  
        "WorkGroup": "dev",  
        "EngineVersion": {  
            "SelectedEngineVersion": "Athena engine version 2",  
            "EffectiveEngineVersion": "Athena engine version 2"  
        }  
    }  
}
回答済み 3年前

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

質問に答えるためのガイドライン

関連するコンテンツ