Kinesis Analytics throws OutOfMemoryError during testing of small, single event

0

The following code is uploaded to Kinesis Analytics application:

from pyflink.datastream import StreamExecutionEnvironment
from pyflink.common.typeinfo import Types
from pyflink.common import Row

env = StreamExecutionEnvironment.get_execution_environment()

def my_map_func(row):
  items = [Row('1'), Row('2'), Row('3') ]
  return Row(items)

ds = env.from_collection(
  collection=[(1, 'aaa'), (2, 'bbb')],
  type_info=Types.ROW([Types.INT(), Types.STRING()]))

ds2 = ds.map(my_map_func, output_type=Types.ROW([
  Types.LIST(
    Types.ROW([
      Types.STRING(),
    ])
  ),
]))

with ds2.execute_and_collect() as results:
  for result in results:
    print(result)

Resulting in the following error when executing on the task managers:

Caused by: java.lang.OutOfMemoryError: Java heap space
	at org.apache.flink.table.runtime.util.SegmentsUtil.allocateReuseChars(SegmentsUtil.java:89)
	at org.apache.flink.table.runtime.util.StringUtf8Utils.decodeUTF8(StringUtf8Utils.java:126)
	at org.apache.flink.table.runtime.typeutils.serializers.python.StringSerializer.deserialize(StringSerializer.java:90)
	at org.apache.flink.table.runtime.typeutils.serializers.python.StringSerializer.deserialize(StringSerializer.java:41)
	at org.apache.flink.api.java.typeutils.runtime.RowSerializer.deserialize(RowSerializer.java:345)
	at org.apache.flink.api.java.typeutils.runtime.RowSerializer.deserialize(RowSerializer.java:72)
	at org.apache.flink.api.common.typeutils.base.ListSerializer.deserialize(ListSerializer.java:137)

I have also reproduced this same scenario in Kinesis Analytics Studio

mattyf
posta un anno fa327 visualizzazioni
1 Risposta
0

Thank you for contacting us. Please note that inorder to understand the reason for the error "java.lang.OutOfMemoryError: Java heap space" we would need to check internal metrics and logs. Thus I request you to please raise a case/ticket with AWS Support with the following information:

  1. Kinesis analytics application arn
  2. Timeframe of issue

Thank you!

AWS
TECNICO DI SUPPORTO
con risposta un anno fa

Accesso non effettuato. Accedi per postare una risposta.

Una buona risposta soddisfa chiaramente la domanda, fornisce un feedback costruttivo e incoraggia la crescita professionale del richiedente.

Linee guida per rispondere alle domande