How to handle numbers with DynamoDB data import from Amazon S3

1

I am using the new DynamoDB data import from Amazon S3 functionality and it looks very powerful however my DynamoDB tables are all being generated with strings for every attribute when I am expecting numbers and booleans for some. For example:

my_string,my_number,my_bool
"ABCDEFG",21,true

...ends up as...

{
	"my_string": "ABCDEFG",
	"my_number": "21",
	"my_bool": "true"
}

...when I was expecting...

{
	"my_string": "ABCDEFG",
	"my_number": 21,
	"my_bool": true
}

Is there a way of instructing the import job to encode my attributes correctly?

Many thanks for any advice

  • How are you generating the data you are using for the import? It uses a DDB json format which has the types used in each field, looks like the data is being specified as a string and hence the items are created with the attributes being strings

demandé il y a 2 ans331 vues
1 réponse
2
Réponse acceptée

Hi, @mgvarley.

Unfortunately, when importing CSV, all attributes other than attributes that can be specified as options such as keys and indexes are imported as DynamoDB strings.
This is the current specification.

https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/S3DataImport.Format.html

So you need to convert the CSV to DynamoDB Json or Amazon Ion before doing the ingestion.

profile picture
EXPERT
iwasa
répondu il y a 2 ans
profile pictureAWS
EXPERT
Chris_G
vérifié il y a 2 ans
  • Thanks for the answer @iwasa this is a shame as it is much faster for us to generate a CSV from a Postgres table (30m rows) than stream out DynamoDB JSON. It would seem logical for this functionality to treat numbers as numbers. Are there any plans on the roadmap to improve this or can we put in a feature request? I suspect there are many users for whom lack of number support would prevent them from using this great new functionality.

  • Do you have a tool to convert CSV to Dynamo DB JSON?

Vous n'êtes pas connecté. Se connecter pour publier une réponse.

Une bonne réponse répond clairement à la question, contient des commentaires constructifs et encourage le développement professionnel de la personne qui pose la question.

Instructions pour répondre aux questions