Help processing Kinessis Records with KCL and Java

0

How am I supposed to process the actual record in Java using KCL?

I'm following the guidance provided https://docs.aws.amazon.com/streams/latest/dev/kcl2-standard-consumer-java-example.html, I can connect to the data stream, I can get the number of records available, however what the example doesn't show is how to actually get the record (Json string). From the example I can see that I can use r.data() to get the record's data, it comes as a read only ByteBuffer, I can convert this to string by using StandardCharsets.US_ASCII.decode(r.data()).toString(), however the resulting string is definitely encoded, I have tried doing Base64 decoding but I get error java.lang.IllegalArgumentException: Illegal base64 character 3f. So what is the simplest way to get the payload? Below is my processRecords method:

public void processRecords(ProcessRecordsInput processRecordsInput) {
    try {
        System.out.println("Processing " + processRecordsInput.records().size() + " record(s)");
        processRecordsInput.records().forEach((r) -> {
            try {
                Decoder dec = Base64.getDecoder();
                String myString = StandardCharsets.US_ASCII.decode(r.data()).toString();
                byte[] bt = dec.decode(myString);
 
            } catch (Exception e) {
                e.printStackTrace();
            }
        });
    } catch (Throwable t) {
        System.out.println("Caught throwable while processing records. Aborting.");
        Runtime.getRuntime().halt(1);
    } finally {
    }
}

From here I can get myString but when I get to bt I get the exception shown. I have not found a single resource explaining how to get the record. I post the record to kinesis using aws kinesis put-record --stream-name testStream --partition-key 1 --data {"somedata":"This Data"}

rhaces
질문됨 2년 전700회 조회
1개 답변
0
수락된 답변

The problem was in the producer which was not producing the correct data, using String myString = StandardCharsets.US_ASCII.decode(r.data()).toString(); would give you the String as expected.

rhaces
답변함 2년 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠