How to handle numbers with DynamoDB data import from Amazon S3

1

I am using the new DynamoDB data import from Amazon S3 functionality and it looks very powerful however my DynamoDB tables are all being generated with strings for every attribute when I am expecting numbers and booleans for some. For example:

my_string,my_number,my_bool
"ABCDEFG",21,true

...ends up as...

{
	"my_string": "ABCDEFG",
	"my_number": "21",
	"my_bool": "true"
}

...when I was expecting...

{
	"my_string": "ABCDEFG",
	"my_number": 21,
	"my_bool": true
}

Is there a way of instructing the import job to encode my attributes correctly?

Many thanks for any advice

  • How are you generating the data you are using for the import? It uses a DDB json format which has the types used in each field, looks like the data is being specified as a string and hence the items are created with the attributes being strings

質問済み 2年前301ビュー
1回答
2
承認された回答

Hi, @mgvarley.

Unfortunately, when importing CSV, all attributes other than attributes that can be specified as options such as keys and indexes are imported as DynamoDB strings.
This is the current specification.

https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/S3DataImport.Format.html

So you need to convert the CSV to DynamoDB Json or Amazon Ion before doing the ingestion.

profile picture
エキスパート
iwasa
回答済み 2年前
profile pictureAWS
エキスパート
Chris_G
レビュー済み 2年前
  • Thanks for the answer @iwasa this is a shame as it is much faster for us to generate a CSV from a Postgres table (30m rows) than stream out DynamoDB JSON. It would seem logical for this functionality to treat numbers as numbers. Are there any plans on the roadmap to improve this or can we put in a feature request? I suspect there are many users for whom lack of number support would prevent them from using this great new functionality.

  • Do you have a tool to convert CSV to Dynamo DB JSON?

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

質問に答えるためのガイドライン