Unable to create Glue Resource Policy

Question

Hi,  
  
I'm looking to ditch AWS Lake Formation as a solution to enable cross-account access to my data lake's Glue Data Catalog, in favour of the much simpler Glue Resource Policy solution.  
  
To be clear, I no longer want Lake Formation to have any effect on my permissions model and want to revert to using pure IAM. However, when I try to create a Glue Resource Policy, AWS complains I need to "EnableHybrid" because it thinks I'm still using LakeFormation in addition to the Glue Resource policy. This is the response I get when running `aws glue create-resource-policy`:  
  
    $ aws glue put-resource-policy --cli-input-json file://policy.json  
  
    An error occurred (InvalidInputException) when calling the PutResourcePolicy operation: EnableHybrid must be set to TRUE to allow policy hybrid.  
  
I've completely deleted every database from my Glue Data Catalog and backed the metadata up locally. This has also removed all data permission grants from AWS Lake Formation. The Lake Formation settings are reverted to the default settings, as shown below:  
  
    $ aws lakeformation get-data-lake-settings  
    {  
        "DataLakeSettings": {  
            "DataLakeAdmins": \[],  
            "CreateDatabaseDefaultPermissions": \[],  
            "CreateTableDefaultPermissions": \[],  
            "TrustedResourceOwners": \[]  
        }  
    }  
  
    $ aws lakeformation list-permissions  
    {  
        "PrincipalResourcePermissions": \[]  
    }  
  
I could set EnableHybrid to TRUE, however I don't think this should be necessary, and I worry that all of the error-proneness of Lake Formation will creep in if it's not completely disabled.  
  
Is there a way to completely revert all Lake Formation settings and have my AWS Account behave as if I'd never touched Lake Formation?   
  
Regards,  
Toby

Answer

Sure enough, after putting the resource policy with "EnableHybrid" set to TRUE, the evil AWS Lake Formation still has its say. This is executing in the "borrower" / "external" account:  
  
    $ aws athena start-query-execution --query-string 'SELECT * FROM datalake.prod.example' --work-group dev  
    {  
        "QueryExecutionId": "REDACTED"  
    }  
  
    $ aws athena get-query-execution --query-execution-id "REDACTED"  
    {  
        "QueryExecution": {  
            "QueryExecutionId": "REDACTED",  
            "Query": "SELECT * FROM datalake.prod.example",  
            "StatementType": "DML",  
            "ResultConfiguration": {  
                "OutputLocation": "REDACTED"  
            },  
            "QueryExecutionContext": {},  
            "Status": {  
                "State": "FAILED",  
                "StateChangeReason": "Insufficient Lake Formation permission(s) on example (Service: AWSGlue; Status Code: 400; Error Code: AccessDeniedException; Request ID: REDACTED; Proxy: null)",  
                "SubmissionDateTime": "2021-08-22T12:02:37.211000+10:00",  
                "CompletionDateTime": "2021-08-22T12:02:48.740000+10:00"  
            },  
            "Statistics": {  
                "EngineExecutionTimeInMillis": 10787,  
                "DataScannedInBytes": 0,  
                "TotalExecutionTimeInMillis": 11529,  
                "QueryQueueTimeInMillis": 641,  
                "ServiceProcessingTimeInMillis": 101  
            },  
            "WorkGroup": "dev",  
            "EngineVersion": {  
                "SelectedEngineVersion": "Athena engine version 2",  
                "EffectiveEngineVersion": "Athena engine version 2"  
            }  
        }  
    }

Unable to create Glue Resource Policy

Relevant content