Is it possible to specify DB snapshot in AWS Lake Formation?
When defining blueprints in AWS Lake Formation, can we specify a particular snapshot? Does Lake Formation always uses the recent snapshot by default?
AWS Lake Formation blueprints make a JDBC connection to your database and effectively do a "select *" on the tables you want to import. It's referred to as a database snapshot because it brings full tables over at a point in time. It does not use RDS backup snapshots to import data.
Lake Formation enables you to create a workflow from a blueprint, creating workflows is much simpler and more automated in Lake Formation. The other option is to create workflows in AWS Glue. Lake Formation provides the following types of blueprints:
Database snapshot – Loads or reloads data from all tables into the data lake from a JDBC source. You can exclude some data from the source based on an exclude pattern. Incremental database – Loads only new data into the data lake from a JDBC source, based on previously set bookmarks. You specify the individual tables in the JDBC source database to include. For each table, you choose the bookmark columns and bookmark sort order to keep track of data that has previously been loaded. The first time that you run an incremental database blueprint against a set of tables, the workflow loads all data from the tables and sets bookmarks for the next incremental database blueprint run. You can therefore use an incremental database blueprint instead of the database snapshot blueprint to load all data, provided that you specify each table in the data source as a parameter. Log file – Bulk loads data from log file sources, including AWS CloudTrail, Elastic Load Balancing logs, and Application Load Balancer logs.
This is not to be confused with the functionality in Amazon RDS that can take data from an RDS backup snapshot and load that into S3. For more information on this see: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ExportSnapshot.html
No, Lake Formation BluePrint read data from table and not from the previously created RDS backup snapshots.
Is it possible to specify DB snapshot in AWS Lake Formation?Accepted Answerasked a year ago
Glue job keep running indefinitely on S3 registered with Lake Formationasked a year ago
Access error : Spark query from AWS EMR with AWS Lake Formationasked 5 months ago
Access denied error while creating Data Filter in Lake Formationasked 6 months ago
InternalFailure Assiging administrator to Lake Formationasked 6 months ago
AWS Lake Formation Data Filter Issueasked a month ago
Using Athena to query AWS Lake Formation databaseasked 2 months ago
Are you able to hide tables in a database using Lake Formation Taggingasked 3 months ago
How to best handle lake formation row level security from a shared IAM roleasked 7 months ago
User defined groups and blue prints in Lake Formationasked 3 years ago