How to handle numbers with DynamoDB data import from Amazon S3

1

I am using the new DynamoDB data import from Amazon S3 functionality and it looks very powerful however my DynamoDB tables are all being generated with strings for every attribute when I am expecting numbers and booleans for some. For example:

my_string,my_number,my_bool
"ABCDEFG",21,true

...ends up as...

{
	"my_string": "ABCDEFG",
	"my_number": "21",
	"my_bool": "true"
}

...when I was expecting...

{
	"my_string": "ABCDEFG",
	"my_number": 21,
	"my_bool": true
}

Is there a way of instructing the import job to encode my attributes correctly?

Many thanks for any advice

  • How are you generating the data you are using for the import? It uses a DDB json format which has the types used in each field, looks like the data is being specified as a string and hence the items are created with the attributes being strings

已提问 2 年前314 查看次数
1 回答
2
已接受的回答

Hi, @mgvarley.

Unfortunately, when importing CSV, all attributes other than attributes that can be specified as options such as keys and indexes are imported as DynamoDB strings.
This is the current specification.

https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/S3DataImport.Format.html

So you need to convert the CSV to DynamoDB Json or Amazon Ion before doing the ingestion.

profile picture
专家
iwasa
已回答 2 年前
profile pictureAWS
专家
Chris_G
已审核 2 年前
  • Thanks for the answer @iwasa this is a shame as it is much faster for us to generate a CSV from a Postgres table (30m rows) than stream out DynamoDB JSON. It would seem logical for this functionality to treat numbers as numbers. Are there any plans on the roadmap to improve this or can we put in a feature request? I suspect there are many users for whom lack of number support would prevent them from using this great new functionality.

  • Do you have a tool to convert CSV to Dynamo DB JSON?

您未登录。 登录 发布回答。

一个好的回答可以清楚地解答问题和提供建设性反馈,并能促进提问者的职业发展。

回答问题的准则