This post describes how you can use KDA/Flink for exactly that use case: https://aws.amazon.com/blogs/big-data/streaming-etl-with-apache-flink-and-amazon-kinesis-data-analytics/. The section "Persisting data in Amazon S3 with data partitioning" describes how to realize data partitioning and you can find the sources on GitHub: https://github.com/aws-samples/amazon-kinesis-analytics-streaming-etl.
- Accepted Answerasked 2 years ago
- AWS OFFICIALUpdated 4 years ago
- How do I stream data from CloudWatch Logs to a VPC-based Amazon OpenSearch Service cluster in a different account?AWS OFFICIALUpdated 2 years ago
- How do I stream log data from CloudWatch Logs to a cross-Region and cross-account Kinesis data stream?AWS OFFICIALUpdated 2 years ago
- EXPERTpublished 2 years ago