How to handle numbers with DynamoDB data import from Amazon S3

1

I am using the new DynamoDB data import from Amazon S3 functionality and it looks very powerful however my DynamoDB tables are all being generated with strings for every attribute when I am expecting numbers and booleans for some. For example:

my_string,my_number,my_bool
"ABCDEFG",21,true

...ends up as...

{
	"my_string": "ABCDEFG",
	"my_number": "21",
	"my_bool": "true"
}

...when I was expecting...

{
	"my_string": "ABCDEFG",
	"my_number": 21,
	"my_bool": true
}

Is there a way of instructing the import job to encode my attributes correctly?

Many thanks for any advice

  • How are you generating the data you are using for the import? It uses a DDB json format which has the types used in each field, looks like the data is being specified as a string and hence the items are created with the attributes being strings

已提問 2 年前檢視次數 330 次
1 個回答
2
已接受的答案

Hi, @mgvarley.

Unfortunately, when importing CSV, all attributes other than attributes that can be specified as options such as keys and indexes are imported as DynamoDB strings.
This is the current specification.

https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/S3DataImport.Format.html

So you need to convert the CSV to DynamoDB Json or Amazon Ion before doing the ingestion.

profile picture
專家
iwasa
已回答 2 年前
profile pictureAWS
專家
Chris_G
已審閱 2 年前
  • Thanks for the answer @iwasa this is a shame as it is much faster for us to generate a CSV from a Postgres table (30m rows) than stream out DynamoDB JSON. It would seem logical for this functionality to treat numbers as numbers. Are there any plans on the roadmap to improve this or can we put in a feature request? I suspect there are many users for whom lack of number support would prevent them from using this great new functionality.

  • Do you have a tool to convert CSV to Dynamo DB JSON?

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南