Local DynamoDB table not visible between jar and cli

0

Hello! When setting some tests to run against a jar, I came across a data visibility problem I don't really understand.

The jar I have can be configured with the port of a local-dynamo to use, as well as a table name, but otherwise uses a hardcoded region & empty static provider:

DynamoDbClient.fromEnvironment {
    region = "us-east-1"
    credentialsProvider = StaticCredentialsProvider{ accessKeyId = ""; secretAccessKey = "" }
    endpointUrl = Url.parse("http://localhost:${port}")
}

The test uses the aws-cli to create a table with some pre-populated data to run against

 - name: dynamodb
    image: amazon/dynamodb-local:1.21.0
    ports:
    - containerPort: 8000
    command: ["java", "-jar", "-Djava.library.path=./DynamoDBLocal_lib", "DynamoDBLocal.jar", "-inMemory"]
sh "aws configure set aws_region us-east-1"
sh "aws configure set aws_access_key_id foo"
sh "aws configure set aws_secret_access_key bar"
sh "aws dynamodb create-table --cli-input-json file://table.json --endpoint-url http://localhost:8000"
sh "java -jar my.jar --local.port=8000"

If I add more aws-cli debug lines, such as listing tables or describing the table, it shows as created. But when the jar runs, it connects to the local dynamo just fine, but there are no tables inside. Turns out, if I change the setup to this:

environment {
    AWS_REGION = "us-east-1"
    AWS_ACCESS_KEY_ID = "foo"
    AWS_SECRET_ACCESS_KEY = "bar"
}
sh "aws dynamodb create-table --cli-input-json file://table.json --endpoint-url http://localhost:8000"
sh "java -jar my.jar --local.port=8000"

... then it works.

The region does need to agree between the jar and aws-cli (which is mildly surprising for a local dynamo, but OK, I can buy that), but the rest of the situation is pretty wild to me. Why does create-table work, but the table it makes is not visible to the jar in the former case, but is visible in the latter?

1 Answer
0
Accepted Answer

The reason you see this is because DynamoDB creates the file to store data under a directory named using your credentials, so for every set of credentials provided, its like a new "account" for DynamoDB local. To overcome that, you can use the sharedDb option to share a single file between all access keys.

  • If you use the -sharedDb option, DynamoDB creates a single database file named shared-local-instance.db. Every program that connects to DynamoDB accesses this file. If you delete the file, you lose any data that you have stored in it.

  • If you omit -sharedDb, the database file is named myaccesskeyid_region.db, with the AWS access key ID and AWS Region as they appear in your application configuration. If you delete the file, you lose any data that you have stored in it.

  • If you use the -inMemory option, DynamoDB doesn't write any database files at all. Instead, all data is written to memory, and the data is not saved when you terminate DynamoDB.

Even when using the -inMemory option, you must still provide -sharedDb to have all access keys share the same in memory location.

Defined in DynamoDB Local Notes

profile pictureAWS
EXPERT
answered a year ago
  • Ahhh, I didn't realize -sharedDb affected -inMemory too. Thanks!

  • One followup question though; Why does the 2nd way in my example work? The access-key-id's are still different between the jar and cli ... unless moving them to environment variables makes them no longer apply and so the cli acts as if it has empty-strings like the jar? But at the same time, it seems like AWS_REGION is perfectly functional.

  • In your specific instance, it could possibly be cause by the absence of key as you use "" an empty string which is possibly treated as null.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions