1 Risposta
- Più recenti
- Maggior numero di voti
- Maggior numero di commenti
1
It is not possible to use a wildcard in the include path when crawling MongoDB data using AWS Glue crawlers. The include path for MongoDB must specify the exact database and collection names.
However, there are a few options to consider:
- Create multiple crawlers - one for each database/collection combination. This allows crawling specific subsets of data.
- Use an exclude pattern to skip certain collections or documents based on patterns. For example, to exclude any collection starting with "temp_", you can use the exclude pattern "temp_*".
- If the databases and collections have a consistent naming pattern, you can programmatically generate the crawler configuration including all the combinations. Then update this configuration periodically as needed.
- Another alternative is to copy the MongoDB data into Amazon S3 using tools like AWS Database Migration Service. You can then crawl the data in S3 using wildcards in the include path.
Contenuto pertinente
- AWS UFFICIALEAggiornata 2 anni fa
- AWS UFFICIALEAggiornata 3 anni fa