1 Answer
- Newest
- Most votes
- Most comments
0
I'm going to answer my own question. At least for our purposes, glue is a complete waste of time. The crawl alone runs into the tens of minutes. Instead, writing some javascript in nodejs, using the aws-sdk package, I have an app that I can schedule to run hourly, that retrieves the transactional data from S3, transforms it and writes it to Aurora in seconds. Perhaps I overlooked some performance tuning in setting up the glue crawler, or perhaps it is a new technology that has not yet matured in terms of performance, but for now, it is not suitable to our needs.
answered 5 years ago
Relevant content
- asked a month ago
- Accepted Answerasked 2 years ago
- AWS OFFICIALUpdated 3 years ago
- AWS OFFICIALUpdated 7 months ago
- AWS OFFICIALUpdated 6 months ago