Questions tagged with Amazon DynamoDB

I am doing a single table design. These are the entities that are related to my question: KITCHEN, MENUITEM, USER, ORDER, and FAVOURITE. **Relationships: ** **one to many:** KITCHEN can have many MENUITEM. https://ibb.co/N95p5rM (screenshot) **one to many:** USER can have multiple ORDERS. https://ibb.co/RpMdtQL (screenshot) **many to many:** USER can have multiple FAVOURITE KITCHENS and a KITCHEN can have multiple USERS make it as FAVOURITE. This is the one that I am struggling to implement it. My access pattern should be query to get a particular user with all the kitchens that are favorites. So the user is a partition key, because I want to query by the user.

For any AWS account, there is an [initial DynamoDB table quota of 2,500 tables per AWS Region](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/ServiceQuotas.html#limits-tables). One can request a service quota increase using the [support page](https://aws.amazon.com/support). Can one assume that requests will always be approved? Can a service design assume an unlimited number of tables? Otherwise, is there a limit to the quota increase for DynamoDB tables?

AWS DataPipeline Is Failed for EMrClusterBackup with Error Resource Not Healthy:- JobFlow retired

Hi, i am using Dynamodb ExecuteStatement with Parameter attribute in nodejs. I am dynamically passing string array to one of the column. But i am getting below error while executing. please suggest the fix. InvalidParameterType: Expected params.Parameters[1].SS[0] to be a string Code Snippet: Parameters: [ { S: <dynamic value 1> }, { SS: [<dynamic value 2>] // eg: l1,l2 }

Hello, I'm trying to establish a connection between DynamoDB and Unity to do CRUD operation on a WebGL platform using AWS mobile SDK for unity. In editor mode we are able to do read and write operation but if we take WebGL build and run it in browser there AWS is not talking with Unity. I tried by creating Lambda function to have a gateway for Unity and AWS DynamoDB to establish connection, but still couldn't achieve it. Please guide me on how I can fix it and over come this hurdle. Thank you, Lakshman

I get a reference error saying event is not defined. So I'm trying to take event and get the attributes from it and save them into dynamo. If i hard code the data, it inputs it into my database. so I know i'm getting the data input. ``` const AWS = require('aws-sdk'); const docClient = new AWS.DynamoDB.DocumentClient(); exports.handler = async (event) => { try { await createItem() return { body: 'Successfully created item!' } } catch (err) { return { error: err } } } const params = { TableName: 'serverlessrepo-AestheticsFormDB-FormDataTable-1M0ZV9SXX6E4S', Item: { formId : event.formId, CreditApp : event.creditApp, FollowUp : event.followUp, Injectables : event.injectables, Marketing : event.marketing, MeetAgain :"event.meetAgain, MeetingRating :"event.scale, NoOfPatients : event.patientCount, ReachOut : event.reachOut, SoloOwner : event.soloOwner } } async function createItem(){ try { await docClient.put(params).promise(); } catch (err) { return err; } ```

I'm sending with ajax, a payload of data and then i have a lambda setup to take that data and parse it into attributes to store into my dynamoDB. this is my github showing my index.js(lambda function) and my jque.js(file that is sending the data to the lambda via api gateway). https://github.com/dbaldw10/DynamoDWLambdaAWS is there a way to achieve this?

Is it possible to somehow filter the API response received after updating an item in DynamoDB? With filter I mean something similar to the ProjectionExpression capability with a query operation, for example. Even though the ReturnValues option does however provide a number of "filtering" options, it does however not permit selective attribute inclusion. Is there perhaps another way to achieve this?

Running into issue with [ExecuteStatement](https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_ExecuteStatement.html). The result returned both "LastEvaluatedKey" and "NextToken" but when we fed it into the next "ExecuteStatement" as part of "input", we keep getting an error "NextToken does not match request". ``` const input = { Statement: '...', Parameter: [...] }; let end = false; let nextInput = input; while (!end) { const { LastEvaluatedKey, NextToken, ...rest } = await new ExecuteStatementCommand(nextInput); LastEvaluatedKey === undefined ? end = true : nextInput = { NextToken, ...input }; } ``` Saw this other [question regarding LastEvaluatedKey](https://www.repost.aws/questions/QUgNPbBYWiRoOlMsJv-XzrWg/how-to-use-last-evaluated-key-in-execute-statement-request) which was helpful. However, the proposed solution does not work as our key value is date based (i.e. pulling data based on a date range). So, the query result ended with numerous duplicate items created on same date.

**Summary**: Dynamo DB commands from the CLI do not work when real credentials are set up. The 'endpoint-url' flag should work around this and recognize that localhost endpoints can be hit with no credentials given the default setup of the AWS Dynamo Docker image. Output of command after setting credentials: `An error occurred (ResourceNotFoundException) when calling the DescribeTable operation: Cannot do operations on a non-existent table` Is there a fix or workaround for this? **System**: MacOS Monterey version 12.0.1, Macbook Pro - M1 - 2020 ``` 'aws --version' -> aws-cli/2.4.11 Python/3.9.10 Darwin/21.1.0 source/arm64 prompt/off ``` **To reproduce**: -- Start from a terminal that does NOT have AWS Credentials set up via environment variables or anything else -- Start up a local Dynamo DB Instance on Docker: ``` docker pull amazon/dynamodb-local docker run -p 8000:8000 --name=ddblocal -d amazon/dynamodb-local ``` -- Create a table: ``` aws dynamodb create-table --attribute-definitions "[{ \"AttributeName\": \"key\", \"AttributeType\": \"S\"}, { \"AttributeName\": \"valueA\", \"AttributeType\": \"S\"}]" --table-name test_table --key-schema "[{\"AttributeName\": \"key\", \"KeyType\": \"HASH\"}, {\"AttributeName\": \"valueA\", \"KeyType\": \"RANGE\"}]" --endpoint-url "http://localhost:8000" --provisioned-throughput "{\"ReadCapacityUnits\": 100, \"WriteCapacityUnits\": 100}" --region local ``` -- Query the table (to prove it works): ``` aws dynamodb describe-table --table-name test_table --region local --endpoint-url "http://localhost:8000" ``` -- Set your real AWS Credentials: ``` export AWS_ACCESS_KEY_ID="<REAL KEY ID HERE>" export AWS_SECRET_ACCESS_KEY="<REAL SECRET KEY HERE>" export AWS_SESSION_TOKEN="REAL TOKEN HERE>" ``` -- Query the table again (This one fails for me - see output above) ``` aws dynamodb describe-table --table-name test_table --region local --endpoint-url "http://localhost:8000" ```

Hello I've a jsp with code (from https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/ScanJavaDocumentAPI.html) : ... ``` <%@ page import="com.amazonaws.services.dynamodbv2.AmazonDynamoDB" %> <%@ page import="com.amazonaws.services.dynamodbv2.AmazonDynamoDBClientBuilder" %> <%@ page import="com.amazonaws.services.dynamodbv2.document.DynamoDB" %> <%@ page import="com.amazonaws.services.dynamodbv2.document.Item" %> <%@ page import="com.amazonaws.services.dynamodbv2.document.ItemCollection" %> <%@ page import="com.amazonaws.services.dynamodbv2.document.ScanOutcome" %> <%@ page import="com.amazonaws.services.dynamodbv2.document.Table" %> <%@ page import="com.amazonaws.services.dynamodbv2.model.AttributeValue" %> <%@ page import="com.amazonaws.services.dynamodbv2.model.ScanRequest" %> <%@ page import="com.amazonaws.services.dynamodbv2.model.ScanResult" %> ... <% AmazonDynamoDB client = AmazonDynamoDBClientBuilder.standard().build(); DynamoDB dynamoDB = new DynamoDB(client); String tableName = "TicketsWalletUsersDev"; // Table table = dynamoDB.getTable(tableName); Map<String, AttributeValue> lastKeyEvaluated = null; do { ScanRequest scanRequest = new ScanRequest() .withTableName(tableName) .withLimit(10) .withExclusiveStartKey(lastKeyEvaluated); ScanResult result = client.scan(scanRequest); for (Map<String, AttributeValue> item : result.getItems()){ System.out.println(item.toJSONPretty()); } lastKeyEvaluated = result.getLastEvaluatedKey(); } while (lastKeyEvaluated != null); %> ``` It gives the error: The method toJSONPretty() is undefined for the type Map<String,AttributeValue> I want the string containing the true json and not the string returned from item.toString() that is : {addresses={M: {mobile+75@nearit.com={M: {requestId={S: 909b1963-0ed4-4ac1-9b71-846239ed5bf6_1583142338893,}, verified={BOOL: true}},}, federico+11@nearit.com={M: {requestId={S: 909b1963-0ed4-4ac1-9b71-846239ed5bf6_1581332986414,}, verified={BOOL: true}},}, federico+signup3@nearit.com={M: {requestId={S: 909b1963-0ed4-4ac1-9b71-846239ed5bf6_1581330211793,}, verified={BOOL: true}},}, federico+33@nearit.com={M: {requestId={S: 909b1963-0ed4-4ac1-9b71-846239ed5bf6_1581330013122,}, verified={BOOL: true}},}, mobile@nearit.com={M: {requestId={S: 909b1963-0ed4-4ac1-9b71-846239ed5bf6_1583162677615,}, verified={BOOL: true}},}, federico+12@nearit.com={M: {requestId={S: 909b1963-0ed4-4ac1-9b71-846239ed5bf6_1581333042026,}, verified={BOOL: true}},}, federico+2@nearit.com={M: {requestId={S: 909b1963-0ed4-4ac1-9b71-846239ed5bf6_1581332941950,}, verified={BOOL: true}},}},}, created={S: 2020-02-05T10:40:17.887Z,}, userName={S: 909b1963-0ed4-4ac1-9b71-846239ed5bf6,}} Thanks!

A DynamoDB table suddenly shows error "Key not found in KeySchema.". We didn't change settings. It worked before. We can't explore items now. Obviously it's a bug. Please fix it.

I would like to deploy a multi tenant web application say `webapp.com` and I would like each customer to have their own subdomain like `app.customer1.webapp.com`, `api.customer1.webapp.com`, `app.customer2.webapp.com`, `api.customer2.webapp.com`, etc. This was each customer has their own API and Web App URL. I would like all web app urls to go to the same instance of the application but load a different configuration for each customer - each customer has their own cognito userpool. Similarly for API but I would need to authenticate based on Bearer Token and Domain used. For the backend I think I will have a dynamodb table to hold the details of each customer. The web app is NextJS app so I think I will use a fargate cluster for both the web app and api (I want to avoid API Gateway and lambda). I will also use CDK for all infrastructure so I might also use it to bootstrap each customer using a stack that consists of Route53 configs and other resources like userpool that is unique to each customer. What is the easiest way to achieve this (preferably with CDK) ?

Greetings! I have an application that utilizes a DynamoDB table to manage counters. When we update the value of the counter, we consume the returned value - this returned value is the old value of the counter. This counter is then used to retrieve a value in an array, it is effectively an index. I currently increment these counters with Lambda functions - it's highly likely that many Lambdas may attempt to update the value of a given counter at the same time. It's important that each Lambda function receives a different value of the counter. Utilizing DynamoDB atomic counters, can I expect that each Lambda function will receive a different value for the counter? Example: * 10 Lambdas attempt to update value of a counter where its current value is 0 * The Update Expression is "ADD count :incr" where ":incr" is 1 * Each Lambdas update is processed * The returned value from each update is unique per Lambda and the values are 0 - 9 * The final value of the counter is 10

Greetings! I've created a DynamoDB table that manages counters. These counters have to be strongly consistent - the workflow currently is to update the counter and read the new value of the counter that is returned by the update. At some point I would like to go multi-region with my application and the next logical step is to convert my existing tables to global tables, but I have concerns that this particular counter table will not work on a global scale. My question is that if I convert this table to a global table and multiple requests across regions attempt to increment this counter, can I expect that it is possible two update requests may return the same value across regions? Unhappy path example: * Counter = 1 * Request to increment counter by 1 in region A and B at the same time * Request succeeds and the returned value for Region A and B is 2 This would be bad, we need Region A to get 2 and Region B to get 3.

I have a dashboard and I do not want to have any access restrictions for Amazon employees. Is it possible? any ANT group that provides access to all Amazon employees?

I am trying to insert data from my mqtt topic into dynamodb, but the tutorial in the aws documentation does not refer to the new console. I can see the mqtt messages, and can extract the required message. I cannot however insert the data into a table. I have written rules, and given the correct roles the right permissions, but am getting nowhere. Please help!!

Hi, I would like to enable dynamodb stream on an existing dynamodb table using cloudformation template. I searched but could not find anything in this regard. I see there is way to enable dynamodb stream for an existing table using console but couldn't find any way to achieve same using cloudformation template. Can you help with this .

Hello, I am doing a simple command to list all the items in my table. However, the data I am getting back is not structured the way I want. I want a simple JSON structure but DynamoDB is turning the results into nested objects. DynamoDB gives me below response: ``` /// What I am currently getting [ { id: { S: '8' }, lastName: { S: 'Perry' }, firstName: { S: 'Matthew' } }, { id: { S: '3' }, firstName: { S: 'Joan' }, lastName: { S: 'Peter' } } ] ``` But I want this: ``` /// What I want [ { id: 8 lastName: 'Perry' , firstName: 'Matthew' }, { id: 3, firstName: 'Joan' , lastName: 'Peter' } ] ``` How can I achieve the later result set. Below is my code: ``` const { ExecuteStatementCommand } = require('@aws-sdk/client-dynamodb') const { ddbDocClient, memberTableName } = require('./client.js') const selectAll = async () => { const params = { Statement: `SELECT * FROM ${memberTableName}`, Parameters: [{ S: '3' }] } console.log(params) return await ddbDocClient.send(new ExecuteStatementCommand(params)); } selectAll() .then(d => console.log(d.Items)) ``` ddbDocClient was created like this: ``` const ddbDocClient = DynamoDBDocumentClient.from(ddbClient); ```

I have successfully republished data from my "lorawan/+/uplink" topic to my new topic "datashow". I am now trying send the data I extracted from the original topic to a DynamoDB. I created a rule named "SensorDB_rule" to select the data from the "datashow" topic and send it to a DynamoDB. The data I receive is listed below. ``` { "output": { "DeviceID": "eui-70b3d57ed004c725", "PayloadData": { "Battery": 100, "Sensor": -27.730278573288206, "alt": 63.10, "lat": 38.789196, "lon": -77.073914, "time": "18:52:43" } } } ``` The rule SQL Statement is listed below ``` SELECT output.DeviceID as Device_ID, output.PayloadData.Battery as Battery, output.PayloadData.Sensor as Sensor, output.PayloadData.alt as Altitude, output.PayloadData.lat as Latitude, output.PayloadData.lon as Longitude, output.PayloadData.time as Time FROM 'datashow' ``` I created a DynamoDB table named "SensorDB" and set "DeviceID" as my Partition Key and "time" as my Sort key. With both set to String ``` DeviceID time ``` I'm using the Default settings for my DB table. and created the table Back in the "SensorDB_rule" I selected my table I just created ("SensorDB") and entered the following ``` Partition key- DeviceID Partition key type- STRING Partition key value- $DeviceID Sort Key- time Sort key type- STRING Sort key value- $time ``` and wrote the message to the column "MyMessageData" finally I created a new IAM role names "SensorDB_role" and attached the following permissions AWSIoTLogging AWSIoTRuleActions AWSIoTEventsFullAccess AWSIoTFullAccess AmazonDynamoDBFullAccess I have double checked all my information but I am still not getting anything inside of my table. Any suggestions on how to fix this? (adding policies, correcting my SQL statement, changing my table settings)

Hi, I have a table in DynamoDB that has approximately 2.2 million entries. I am trying to use the AWS CLI to scan 10 thousand entries from the table. However, the following command: ``` aws dynamodb scan --limit 10000 --table-name <table_name> ``` Only returns around 4 thousand entries, with a LastEvaluatedKey. I also tried it with a query command, and I get similar results. Does anyone know why it does not return 10 thousand entries, and how I can fix it? Thanks.

We are working for FHIR(Fast Healthcare Interoperability Resources). We have followed “FHIR works on AWS” and deployed the CloudFormation template given by AWS in our AWS environment.Following is the template that we have deployed https://docs.aws.amazon.com/solutions/latest/fhir-works-on-aws/aws-cloudformation-template.html Requirement : we want to maintain client specific/customized ids as primary key in the server. Problem : server not allowing us to override or mainain client specific (customized ) ids as primary key .Infact , in the runtime, it is generating its own ids and ignoring the id given by us.

Hello, I am sending data using MQTT test client in AWS IoT. The data is showing up properly when I subscribe to the topic. I then wanted to send the data to DynamoDB using a rule. I created the rule and double checked everything: made sure the topics were the same, the rule and MQTT were in the same location (ie: u-west-1), i tried creating different roles when the first didn't work, and i double checked to make sure the attributes were spelled correctly in the MQTT message and DynamoDB table. Any help or additional troubleshooting methods will be greatly appreciated. Thanks.

What is the best way to sync my DynamoDB tables to S3, so that I can perform serverless 'big data' queries using Athena? The data must be kept in sync without any intervention. The frequency of sync would depend on the cost, ideally daily but perhaps weekly. I have had this question a long time. I will cover what I have considered, and why I don't like the options. 1) AWS Glue Elastic Views. Sounds like this will do the job with no code, but it was announced 18 months ago and there have been no updates since. Its not generally available, and there is not information on when it might be. 2) Use dynamodb native backup following this blog https://aws.amazon.com/blogs/aws/new-export-amazon-dynamodb-table-data-to-data-lake-amazon-s3/. I actually already use this method for 'one-off' data transfers that I kick-off manually and then configure in Athena. I have two issues with this option. The first is that, to my knowledge, the export cannot be scheduled natively. The blog suggests using the CLI to kick off exports, and I assume the writer intends that the CLI would need scheduling on a cron job somewhere. I don't run any servers for this. I imagine I could do it via a scheduled Lambda with an SDK. The second issue is that the export path in S3 always includes a unique export ID. This means I can't configure the Athena table to point to a static location for the data and just switch over the new data after a scheduled export. Perhaps I could write another lambda to move the data around to a static location after the export has finished, but it seems a shame to have to do so much work and I've not seen that covered anywhere before. 3) I can use data pipeline as described in https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBPipeline.html. This post is more about backing data up than making it accessible to Athena. I feel like this use case must be so common, and yet none of the ideas I've seen online are really complete. I was wondering if anyone had any ideas or experiences that would be useful here?

I've observed that the value of the AWS Step Functions context variable $$.State.EnteredTime will trim trailing zeros when the time is "on a second" (i.e. there are no milliseconds) rather than using ".000". This is okay according to ISO 8601, so within the realm of correctness for the Step Functions team. However, the DynamoDB .Net SDK is unable to deserialize the date in that format. So, when using a DDB SDK integration in a StateMachine that writes out the value of $$.State.EnteredTime (unaltered) to a table, attempting to deserialize it using a DynamoDBContext, will result in an exception being thrown. Is this a bug, and if so in which product? IMHO, the trimming of trailing zeros is a weird and essentially pointless optimization (if that's the motivation), but the brittleness of the DDB .Net SDK is likewise an issue. More importantly, is there a better way to handle this other than writing a custom IPropertyConverter and using it on any property that may be updated from a StateMachine?

I use nodejs and aws-sdk to use dynamicodb. It's weird using putitem. I was successful in the result of Putitem. However, if I check the contents of the table, only a part of the data is included. The table contains exactly 100 items normally, and only some of the data that exceeds 100. Similar results were obtained by using batchWrite instead of putitem. What more should I check?

Hi everyone! I'd like to understand better the billing composition regardless of AWS Backup on DynamoDB resources since I got an unexpected increase in my billing. I'm aware of AWS Backup billing itself thanks to the [documentation](https://aws.amazon.com/backup/pricing/), anyway, when I access the Billing service I can notice an exponential billing pricing in DynamoDB service, on the section `Amazon DynamoDB USE1-TimedBackupStorage-ByteHrs` the description allows me to see that I'll be paying $0.10 per GB-month of storage used for on-demand backup, showing me that I've used 14,247.295 GB-Month (This makes sense with the billing I got) but where my doubt comes from is, **where does all those GB come from?** The last snapshot-size just shows 175.5 GB I've configured my backup plan with the following parameters: ``` { "ruleName": "hourly-basis", "scheduleExpression": "cron(0 * ? * * *)", "startWindowMinutes": 60, "completionWindowMinutes": 180, "lifecycle": { "toDeletedAfterDays": 30 } } ``` I'm also copying snapshots into a second region on `us-west-2` As you can see, I'm handling a schedule expression on an hourly basis backup because of compliance requirements. *Is this enough justification for the high billing?* I'm aware that backups with low RPO are commonly expensive but I just want to be sure that this billing is not higher than it should be because of any wrong Backup configuration. Thanks in advance!

Hi, **TL;DR: Can anyone give a recommend approach to storing customisable n-level hierarchy trees for grouping and navigating results via a frontend Amplify-powered web app (using DynamoDB or any other database solution that can be mapped to AppSync ideally)?** **Some background** I'm building an multi-tenant IoT analytics solution that takes data from some sensors out in the field, uploads to AWS, processes this data and stores in a DynamoDB table (i.e. a very "standard" setup). I'm planning on adding a web frontend (built using Amplify and an AppSync GraphQL layer) that will allow users to navigate a **customisable, n-level** hierarchy tree of assets, in order to view the sensor data we've collected. Examples of valid hierarchies include: Country -> Site -> Building -> Floor -> Room -> Sensor (6-level) or Site -> Building -> Room -> Sensor (4-level) etc. The important thing here, is that this hierarchy tree can differ per customer, and needs to be customisable on a tenant-by-tenant basis, but we don't need to do any complex analysis or navigation of relationships between hierarchy levels (so, to me, something like Amazon Neptune or another graph database feels a bit overkill, but perhaps I’m wrong). My first thought was to try and build a hierarchical relationship inside of a DynamoDB table, possibly making use of a GSI to provide this, but in all of the examples I’ve seen online, the focus is very much on quick retrieval, but not so quick updating of hierarchy trees – now, whilst it’s unlikely that these tree structures would be updated on a regular basis, it is something we need to be able to support, so the idea of possibly updating ‘000s of rows in DynamoDB every time we want to make a change to the hierarchy tree for a given control area doesn’t seem quite right to me. Hence, my question above. I'm ideally looking for guidance on how to structure a DDB table to best support BOTH optimal retrieval of, and updates to, hierarchy trees in our application, but if DDB isn't the right answer here, then suggestions of alternatives would also be greatly appreciated. Many thanks in advance.

Hi, my Developer propose for my Video on Demand app for the search function Angolia or Open Search. He is telling me that DynamoDB is not perfect for the Filter. Is it true? What would be the best and the cheapest way?

I'm working on an online turn-based strategy game called CivPlanet. There will be dozens of CivPlanet games running at once, each with its own unique gameID. The gamestate consists of several uniquely named JSON objects, henceforth called CivPlanet objects. Players interact with the server via REST requests, which are sent somewhat infrequently. When the player plays the game, they're probably sending around 1 request per minute. In short, I need a database service to store all CivPlanet objects. Each CivPlanet object has a timestamp, gameID, and a name to distinguish it from other objects in that game. CivPlanet objects are never created or deleted once the game is added to the database. However, some can be modified. Also, not all games will share the same set of objects. I need to be able to: * Retrieve a list of all CivPlanet games, along with some metadata about each game such as whether it is accepting new players. * Conditionally retrieve all CivPlanet objects associated with a given gameID that were updated after a given timestamp. * Lock all CivPlanet objects under a given gameID as I prepare to update the gamestate. * Atomically overwrite certain CivPlanet objects that share a gameID. * Release the lock. The server receives a mixture of queries and events from the player. When responding to a query, the server needs to check for any updated data on the database, then process the query and return the result. When responding to the event, it needs to obtain a lock, check for updated data, and process the event. If the event fails, it releases the lock and notifies the player. If it succeeds, it publishes the new data to the database and releases the lock, then notifies the player. My question is, what database service is best suited for these requirements, and what structure should I use within that service? I was looking at DynamoDB. I thought gameID could be the partition key. I'm not sure if I need a sort key, or what that sort key would be. I probably need two databases, one that maps from gameID to metadata, and another that maps from gameID+objectName to JSON. Any thoughts?

I am migrating data from a MySQL DB to DynamoDB. I am using object-mapping rule to transform the data to match my needs. However I have a EPOCH timestamp stored in MySQL DB that I would like to transform to Datetime before loading into my DynamoDB. I see there is a transformation rule I could use to by leveraging SQLite expressions. However I couldnt use transformation rules in conjunction with object-mapping. Is there any other way I could transform my date field by still using 'object-mapping' rule ?

Hi everyone, I have an ECS cluster running and one of service container hits DynamoDB with GETITEM request. Most of these requests get completed within 10 ms but every 30 minutes or so I can see latency going over 2+ seconds. I am using Java SDK to hit the DynamoDB. Intially I thought that this might happen due to connections creation latency so I am using keep alive connection now but still facing same issue. To get more depth I started listening to request metrics and pushed them on to Kibana. Below is a sample request metric : ``` { "HttpRequestTime": 158.081, "RequestSigningTime": 15.361, "CredentialsRequestTime": 2184.899, "ResponseProcessingTime": 17.205, "totalTime": 2707.05, "responseCode": 200, "ClientExecuteTime": 2707.003, "consistentRead": false, "RequestMarshallTime": 198.897, "ApiCallLatency": 242.628, "HttpClientReceiveResponseTime": 22.391, "HttpClientSendRequestTime": 2.166 } ``` As you can see that totalTime taken by GETITEM request is 2707 and CredentialsRequestTime is taking most of the time. So how can I minimize this?

I hope we're asking this in the right area. We are very new to AWS. We have an EC2 server with WHM running two websites. One using MySQL. The other is in development and the developer is suggesting we use MongaDB. We have developer saying we cannot run both on the same server, the other developer saying we can. We looking on feedback and what might be the best server structure to use. Ideally we'd like to run both from one server. How would we best achieve this, if at all? Thanks.

According to the documentation for [ExecuteStatement](https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_ExecuteStatement.html): > If `LastEvaluatedKey` is present in the response, you need to paginate the result set. How can I use `LastEvaluatedKey` if `ExecuteStatement` request has no `ExclusiveStartKey` parameter as `Scan` and `Query`. I cannot use `NextToken` because it is not returned when `Limit` is specified.

I created a Athena connector for my DynamoDB. However, when running a simple select query, I run into the following errors. Any help will be appreciated! GENERIC_USER_ERROR: Encountered an exception[Runtime.ExitError] from your LambdaFunction[arn:aws:lambda:us-west-1:541750221373:function:my_test_catalog] executed in context[retrieving meta-data] with message[RequestId: 8af8027b-2385-4279-ac6c-d45773912907 Error: Runtime exited with error: exit status 1] This query ran against the "default" database, unless qualified by the query. Please post the error message on our forum or contact customer support with Query Id: 542232b7-3070-4793-a6f5-78a653164712

If we use only PartiQL for Dynamodb calls, do the calls get automatically routed via DAX because we are using DAX cache also in front of some of our tables. Basically how to use PartiQL along with DAX Thanks

Hi, I have a appsync Schema which has composite key @key(name: "FinderJobbyDateWithCompositKey", fields: [ "finder_id", "date", "job_status" ], queryField: "JobsFinderWithCompositKey") I did amplify push and it successfully created GSI with the respective table. the table has the data but when I run query query MyQuery { JobsFinderWithCompositKey(finder_id: "+929898989898", dateJob_status: {eq: {date: "2022-04-06T09:00:00.000Z", job_status: "15"}}) { items { cod_charge createdAt } } } it give me Null in response. Please specify what is the issue and how to solve and get result. I followed this link: https://docs.amplify.aws/cli-legacy/graphql-transformer/key/#example-get-orders-by-customer-email-by-createdat

Hi, I am using AWS Lamba to read data from web and store it in dynamoDB at periodic intervals during the day. Post which, I take the daily dump from DynamoDB and get a CSV file is S3 for daily reporting. This had been working perfectly fine and I was getting data for all batches during the day. But off late my S3 dump seems to have gaps and I am missing data for different batches. And these missing batches are different for different days. I am not getting any error message - so, just wanted to get some guidance on how to go about debugging this. Below are stats for my dyanmoDB table: Item count: 235,267 Table size: 65.6 megabytes Average item size: 279.03 bytes Provisioned read capacity units: 1 Provisioned write capacity units: 1 Also below is snapshot of key operational metrics for the table: https://ibb.co/TPhWqBj https://ibb.co/jVgRS93 Regards, Dbeings

I'm using dynamoDB to save data from multiple devices and I noticed that only the last connected device that sends the data, never all simultaneous. Then I saw that it was because of the certificates and that I would have to register the devices one by one. But it turns out that I will use more than 10000 devices and it will be almost impossible to register one by one. So if I had a certificate that worked for all devices it would help a lot.

I am writing some code to query Dynamo with a certain filter and I'm using the `expression` package in the Go sdk: The variable `project` that is referred to is defined in another section of the code. Its string properties `owner` and `name` are not nil/empty. ``` dynamo := dynamodb.New(sess) filt := expression.Name("projectOwner").Equal( expression.Value(project.owner), ).And( expression.Name("projectName").Equal( expression.Value(project.name), ), ) expr, err := expression.NewBuilder().WithFilter(filt).Build() if err != nil { panic(err) } queryInput := dynamodb.QueryInput{ KeyConditionExpression: expr.KeyCondition(), ProjectionExpression: expr.Projection(), ExpressionAttributeNames: expr.Names(), ExpressionAttributeValues: expr.Values(), TableName: aws.String("projects"), } out, err := dynamo.Query(&queryInput) if err != nil { panic(err) } ``` I get this error: ``` panic: ValidationException: ExpressionAttributeNames can only be specified when using expressions ``` I don't know where it's from or what it means: I *am* using expressions.

I have a use-case where I have to read DynamoDB table data and convert this into csv file and write into a s3 bucket using AWS lambda function. DynamoDB tables data is bit low around 2-3 MB so I want to read entire table every week. I am new to AWS and services and want to how to read table without triggers and how to schedule lambda function so it can run every week? Please share the steps to accomplish this task. Any help would be appreciated.

I have the following Graphql schema with relationship: ``` type Product @model{ id: ID! @primaryKey name: String! img: String! components: [Component]! @hasMany categories: [Category]! @hasMany } type Component @model{ id: ID! @primaryKey name: String! } type Category @model{ id: ID! @primaryKey name: String! } ``` A product can have multiple components and multiple categories. I would like to have a mutations that can allow me to create a Product and also the components and categories associated. Amplify codegen already created the mutation "CreateProduct" but it don't allow me to specify the component and category data in order to create them and link to the product. It seems that i must create the product first and then the component and category separately. I need to write a custom mutation in order to do what i want or it's impossible? If it's possible with a custom mutation, how can I write it? Thank you!

I am seeing this error intermittently "Timeout waiting for an available tube" from our DAX cluster. Its three-node cluster. What could be the issue.

We are planning to use DynamoDB global tables to enable automatic replication to our failover region. Our primary region is us-east-1 where several of our processes need to run because that is the only instance of the CloudFront control plane. In the event of a us-east-1 outage, we will redirect to our failover region, which will be making changes to the DynamoDB instance there. When us-east-1 comes back online, I'm expecting replication to kick in and make the necessary changes there, right? Now here's the tricky part. We initiate background tasks (which must run in us-east-1, or at least when us-east-1 is alive) based on changes to the data via DynamoDB streams. When replication happens in us-east-1, will those streams be triggered just as if a service had updated the database?

Hi, I am new to AWS Amplify and would like guidance on how to send a query to an ***existing*** GraphQL API on AWS AppSync. I am unsure how to start as a lot of Amplify coverage creates a *new* AppSync API using the Amplify CLI. ## Objectives * Set up a Node.js project to work with an existing AWS AppSync API, using AWS Amplify as the GraphQL client. * Send a single query to an existing AWS AppSync API. The query lists game results from a DynamoDB table and is called `listGames` in my GraphQL schema. * I need to repeat the query in order to fetch all available database records that satisfy the query. This would mean adding results to an array/object until the `nextToken` is `null` (i.e. no more records can be found for the query). ## Context * This application is deployed in an Amazon ECS container using AWS Fargate. * The ECS service is fronted by an Application Load Balancer (ALB). * A leader board web page fetches game results through a `POST` request to the ALB's DNS name / URL and adds them to a HTML table. ## Notes * For now, API key is my authentication method. I would soon like to switch to a task IAM role in ECS. * The ECS deployment described in 'Context' is working but it sends `POST` requests without AWS libraries. It is my understanding that I would need to use an AWS library in order to use an IAM role for AppSync authentication (used as a [task IAM role in ECS](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-iam-roles.html)). Please correct me if I am mistaken. I would greatly appreciate any help you can give me. Thank you for your time!

Hi, we have a appsync setup with dynamodb which was setup using Amplify GRAPHQL API over a year ago. It's in the us-east-1 region and I access the endpoints from the Philippines (Southeast Asia). We've always had fairly slow response times (around 1sec or more) ever since but I thought it was pretty normal. We're trying to now optimize our app and I read that normal response times should never exceed 500ms. Can someone help us here? Maybe take a look at how we setup our code and speed queries up Thank you very much.

I have this data in our DynamoDB: ``` "stateHistory": { "L": [ { "M": { "state": { "S": "confirmed" }, "timestamp": { "N": "1628579118359" } } }, { "M": { "state": { "S": "active" }, "timestamp": { "N": "1628579118988" } } }, { "M": { "state": { "S": "finished" }, "timestamp": { "N": "1628579251554" } } }, { "M": { "state": { "S": "archived" }, "timestamp": { "N": "1628583091657" } } } ``` And now we want to put all of those states into a txt output file. We tried it like this: aws --profile odm-ss dynamodb scan --table-name booking.history_booking --query "Items[*].[bookingId.S,(stateHistory.L.0.M.state.S)] " --output text> data.txt But get this error: Bad value for --query Items[*].[(stateHistory.L.0.M.state.S)]: Expecting: ['quoted_identifier', 'unquoted_identifier', 'lbracket', 'lbrace'], got: number: Parse error at column 26, token "0" (NUMBER), for expression: "Items[*].[(stateHistory.L.0.M.state.S)]" ^ Any suggestions on how to fetch the states from this map using AWS CLI?

``` session = requests.Session() session.auth = AWS4Auth( # An AWS 'ACCESS KEY' associated with an IAM user. AWS_ACCESS_KEY_ID, # The 'secret' that goes with the above access key. AWS_SECRET, # The region you want to access. AWS_REGION_NAME, # The service you want to access. 'appsync' ) # As found in AWS Appsync under Settings for your endpoint. APPSYNC_API_ENDPOINT_URL = APPSYNC_ENDPOINT # Use JSON format string for the query. It does not need reformatting. query = """query MyQuery { getCustomer(id: "38c76f4f-c495-42e6-8599-647738a12692") { first_name } }""" # Now we can simply post the request... response = session.request( url=APPSYNC_API_ENDPOINT_URL, method='POST', json={'query': query} ) print(response.text) ``` I want to be able to Read/Write data from AWS AppSync's dynamodb through python. I came across this solution online. But when I run it gives me the following error: ``` { "errors" : [ { "errorType" : "UnauthorizedException", "message" : "You are not authorized to make this call." } ] } ``` I have given Administrator Access to the IAM User. What am I doing wrong here?

I have a state that updates DynamoDB by shifting the first word off a list of words. It returns UPDATED_OLD so that the word can be grabbed in the ResultSelector block as index 0 and passed on to the next step. If there are words in the list this works fine, but if the list is empty, there is nothing at index 0 and Step Functions throws a States.Runtime error and the execution fails. I added a Catch block but it isn't catching as expected. Can this error be caught? Thank you.

Hi There, I'm sourcing data from dynamo db and residing into s3 bucket using AWS data pipeline. I have been running this pipeline once in a week to get up-to date the records from dynamo db table. Everything is working fine there's no issues with pipeline since i have small tables i would like to pull all records every time. The thing is that when AWS data pipeline writes the exported files in chunks into s3 which becoming hard now because i have to read the file one by one i don't wanna do that.. I am pretty new with AWS data pipeline service... Can someone guide me how to configure AWS pipeline so that it produces just one output file for each table? or any other better to resolve this? Any help would be appreciated...!

Hi I'm new using DynamoDb and i have one question. How i can access my table located in the cloud of DynamoDb with PHP?

Hi, I'm doing some load testing on my serverless app and I see that it is unable to handle some higher loads. I'm using API Gateway. Lambda(Java 8) and DynamoDB. The code that I'm using is the same as this from this [link]([https://github.com/Aleksandr-Filichkin/aws-lambda-runtimes-performance/tree/main/java-graalvm-lambda/src/lambda-java). In my load testing, I'm using Gatling. The load that I configured is that I'm doing a request with 120 users, then in one minute I ramp users from 120 to 400, and then for 2 minutes I'm making requests with 400 constant users per second. The problem is that my stack is unable to handle 400 users per second. Is it normal? I thought that serverless will scale nicely and will work like a charm. Here is my Gatling simulation code: ```java public class OneEndpointSimulation extends Simulation { HttpProtocolBuilder httpProtocol = http .baseUrl("url") // Here is the root for all relative URLs .acceptHeader("text/html,application/xhtml+xml,application/json,application/xml;q=0.9,*/*;q=0.8") // Here are the common headers .acceptEncodingHeader("gzip, deflate") .acceptLanguageHeader("en-US,en;q=0.5") .userAgentHeader("Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:16.0) Gecko/20100101 Firefox/16.0"); ScenarioBuilder scn = scenario("Scenario 1 Workload 2") .exec(http("Get all activities") .get("/activitiesv2")).pause(1); { setUp(scn.injectOpen(atOnceUsers(120), rampUsersPerSec(120).to(400).during(60), constantUsersPerSec(400).during(Duration.ofMinutes(2)) ).protocols(httpProtocol) ); } } ``` Here are the Gatling report results: [Image link](https://ibb.co/68SYDsb) I'm also receiving an error: **i.n.h.s.SslHandshakeTimeoutException: handshake timed out after 10000ms ** -> This is usually approx 50 requests. It is happening when Gatling is starting to inject 400 constant users per second. I'm wondering what could be wrong. It is too much for API Gateway, Lambda and DynamoDB?

I'm looking for a full example of how to create a dynamodb table based on current relational data. I've been working with relational databases for 25 years and I just can't get my head around how to create a dynamodb table to contain this data, I've gone through the best practice example on AWS, but this doesn't actually show the syntax for creating the table, and it's this bit I'm struggling with. I need the CLI syntax for now, not how to do it at the console although this might be useful in the future. An example with just two or three relational tables will be enough, just to give me a starter and hopefully aid my understanding. Thanks in anticipation.

How to trigger cloudwatch event from lambda upon creation of a record in database. My use case is I want to run a cloudwatch event that is scheduled for 5 mins for one time. As soon as a record is created in DynamoDB by Lambda, this cronjob should be triggered by same lambda and which will invoke target function after 5 mins as scheduled in cronjob.

I am using AWS version 2 and using DynamoDbEnhancedClient for dynamodb operations. I want to have a primary key which should be auto-generated. I see that in previous version of DynamoDB, we have the option of using DynamoDBAutoGeneratedKey annotation for this. But I am not able to find way to configure a auto-generated key with AWS version 2 (DynamoDbEnhancedClient ) Can somebody guide me please. Thanks, Satish

There are two typical uses for my new database. 1. A user will access their record/item via their userID, and they will manage their info and a number of devices, each with a unique devID. 2. A device connects and using its devID, will find the owning userID, then takes action based on attributes in the user item. Two options I could use, each with a single DynamoDB table. A. The table has items that are users or devices, with a partition key of ID and sort key of itemType. User type items have associated attributes like addresses, account and profile info, etc, and a set of deviceIDs. Device type items have associated attributes like their preferences, their type, their capabilities, and an attribute for the userID that owns it. You can access both users and devices really quickly. If you are doing (1) you lookup and find a user, then you will have to use a set attribute that lists the one or more deviceIDs it owns, and then make individual lookups for each device. That's 2 lookups for a user that owns one device and more for multiple devices. Or if you are doing (2), search and find a device, you grab its userID attribute and then lookup the userID item. That's 2 lookups. B. I could reduce the multiple lookups this way: Still one table, but all entries in the table are more homogenous: Every item includes all user related attributes, and includes one device's attributes. The provisioning key is the userID and sort key is deviceID, another indexed attribute is just the deviceID. If you are doing (1) then you lookup the userID and you get one or more records depending on whether they own one device or more. If you are doing (2), then we quickly find the device, and that same item includes all the user info we need and we don't need to do another lookup. The problem with B is that I am duplicating a lot of data about the user in each of the items. Keeping them all synced is going to be problematic too, but that's a lot rarer. So, am I overthinking the lookup costs, and should just go with the multiple lookup as in A, or is the multiple lookups going to be expensive enough that I need to have a better data design?

Hi, During building a simple serverless CRUD API using API Gateway, Lambda(Java 8) and DynamoDB. I came across a problem related to a slow response to the first DynamoDB request. Generally, responses are in the 10s of milliseconds however when I'm executing my lambda function for the first time I am seeing response time around 7s. In my lambda function I'm using Java SDK v2 I also used all the practices from this post: [https://aws.amazon.com/blogs/developer/tuning-the-aws-java-sdk-2-x-to-reduce-startup-time] I did some tests and here are the results of the Lambda execution with DynamoDB request: --- || Avg Response Time | | --- | --- | --- | |Lambda 512mb| 7s | |Lambda 1024mb| 5s | |Lambda 2048mb| 3.5s | |Creating only DynamoDB Client 512mb Lambda | 3s | Note: All of these responses are from cold start Lambda. The avg time for the cold start of my Java Lambdas is 1.5s. Are there any official resources that are saying that the first DynamoDB request will be so slow? Thanks

I have a dynamo table, and have two main types of items: groups and users. I am using groupIds and userIds as partition keys, and userIds and groupIds as sort keys. In this way I am able to map many to many relationships. So querying a userId I can get all of the users groups, and querying a group I can get all of the users in that group. I'm wondering if there is a way to architect it in such a way where I can easily query all groups without performing a scan? My thought is to have a partition key that is just something like "groups" and a sort key of groupIds. Is this a proper approach?

I am considering ways to archive items to S3 using dynamoDB TTL. According to [this official AWS blog ](https://aws.amazon.com/blogs/database/automatically-archive-items-to-s3-using-dynamodb-time-to-live-with-aws-lambda-and-amazon-kinesis-firehose/), the architect is **DyanmoDB Stream => Lambda => Firehose => S3. ** Why not directly writing from Lambda to S3? ** **DyanmoDB Stream => Lambda => S3 **** Thank you!

Hi, I´m doing some reasearch to find the best place to centralize lots of data logs generated by my application considering pricing ,performance and scalabilty. Today all my application data including logs are stored on an Oracle database, but I´m thinking to move all the LOG related data outside it to reduce it´s size and not to worry about storage performance etc... Just put everything on a "infinite" storage apart from my actual database using CDC or a regular batch process **Below are some needs:** - Only inserts are necessary (no updates or deletes) - Customers will need access to this historic data - Well defined pattern of access (one or two indexes at maximum) - Latencies of few seconds is ok - Avoid infrastrucure, DBA, perfomance bottleneck log term... - Infinite Retentiton period (means I don´t want to worry about performance issues, storage size in long term. But something that can handle a few terabytes of data ) **Use case example: ** Historical Sales order by items ( id_item | id_customer | qty_sold | date_inserted ... ), aprox 50 millions records per day Where I would need to see the historical data by item, and by customer for example (two dimensions) I´ve done some research with the options below **S3 + Athena **-> Put everthing on s3, no worries about infrastructure perfomance issues, however as I need query by item and customer, probably it´would be necessary to break files by item or customer , generate millions of partitions to avoid high costs searching on every file etc.. **Postgre** -> Not sure if could be performance bottleneck once tables gets too big even with partition strategies **DynamoDB **-> Not sure if it´s a good alternative to historical data regarding pricing once seconds latency is ok **MongoDB/ DocumentDB **-> Not very familiar with it (I´d prefer SQL language type) but I know it´s has a good scalability **Cassandra**-> dont´know very much **Timeseries db as influxDB, timestream etc..**-> dont´know very much, but it seems appropriate for timeseries What option would you choose ? Sorry in advance if I saying something wrong or impossible :) Thank you!

Is there a way to limit the number of items inserted in a DynamoDB by user ID. I have an alexa skill that i want to limit the number of items added to table by the alexa user ID? Or is there a way to have DynamoDB table count the unique user ids and limit insert for each id after, for example, 5 entries? So, every unique user id can only have 5 entries.

Hi, I started using RDS under AWS Free tier. As per documentation, I was expecting 750 hours of RDS single instance availability without any charge. Below are the details of the charge from billing. I had a basic test table with <100 rows of data. Is there some config change that I need to avoid this charge. RDS $0.115 per GB-month of provisioned gp2 storage running MySQL4.684 GB-Mo$0.54 $0.115 per RDS GB-month of provisioned GP2 storage3.006 GB-Mo$0.35 Similarly for Cloudwatch I am getting charges without specifically activating any alarm. Below is an alarm for my cloudwatch. My provisioned capacity range from 1-10 read / write per second. So not clear how is the below alarm applicable and why is there a charge for this alarm? Do I need to make some changes to DynamoDB & CloudWatch config? In alarm 2022-03-26 22:44:22 ConsumedWriteCapacityUnits < 30 for 15 datapoints within 15 minutes Regards, DBeings

DynamoDB currently does not allow FilterExpression on your GSI hash or sort keys. Hash or sort key should be specified using KeyCondition only. I understand this is made to protect us from making mistakes, but it also removes the ability of doing some pretty good things. As an example, I am working on a table that has relationships from users to resources. In a simplified example, imagine: ``` PK|user#SoMeId SK|resourceA#rEsOuRcEiD NAME|-nice-slug-name- ``` I want to be able to search in "NAME" using "contains", and I would also like to have my list sorted by it, as it is more natural for users than random ids. If I could use a sort key in both KeyExpression and FilterExpression, I could create a GSI using PL/NAME, append prefix to name as well, and have a nice search index. ``` PK|user#SoMeId SK|resourceA#rEsOuRcEiD NAME|resourceA#-nice-slug-name- query (PK=user#SoMeId, SK=begins_with(resourceA), INDEX=pk-name-index, FILTER={ name: contains(slug) }) ``` That would be very nice. Judging by I understand DynamoDB works, it seems like this should be completely possible from the technical perspective. Why don't you guys force an extra argument to enable functionality? Instead of outright banning it, just offer the ability of setting something like `AllowKeyFilter=true` on the API calls, just to make it an obvious opt-in feature, so no one can complain if they pay more than expected in case they made something wrong? When I asked this question on the past, I have been told to duplicate fields, but that's just cumbersome. Why not allow it to be used by those that opt-in? (Continued from https://forums.aws.amazon.com/thread.jspa?threadID=172530)

I have DynamoDB, do ETL to S3 in AWS Glue. What I did: 1. I have added 6 tables to a job and then in Athene executed query for each table select * from t; they work fine. 2. Then I have added 7th table and executed queries but now on each table I have received the error: HIVE_BAD_DATA: Field _lastchangedat's type BINARY in parquet file s3://datawarehouse.glue.extract.output/run-AmazonS3_node1647877440125-16-part-block-0-r-00003-snappy.parquet is incompatible with type bigint defined in table schema Yes, each table has the field _lastchangedat and it mapped to long in output table. The question: how find the reason of this error? How to fix the error?

Hello, I seem to be having a strange issue when querying on my GSI that as soon as I specify the fields I wish to return in the query, I get the "Unsupported type passed" error. Here's the query: ``` { TableName: 'RatingsTable', IndexName: 'username', KeyConditionExpression: '#username = :username', ExpressionAttributeNames: { '#username': 'username' }, ExpressionAttributeValues: { ':username': { S: 'jonathanmsifleet' } }, ProjectionExpression: 'username, imdb_title_id' } ``` The table: ``` RatingsTable: Type: AWS::DynamoDB::Table Properties: TableName: RatingsTable BillingMode: PAY_PER_REQUEST AttributeDefinitions: - AttributeName: imdb_title_id AttributeType: N - AttributeName: username AttributeType: S KeySchema: - AttributeName: imdb_title_id KeyType: HASH - AttributeName: username KeyType: RANGE GlobalSecondaryIndexes: - IndexName: username KeySchema: - AttributeName: username KeyType: HASH Projection: ProjectionType: ALL ``` The strange thing is that if I remove `ProjectionExpression` from the query it executes fine, yet as soon as I specify fields then it throws the error. The error in its entirety in Cloudwatch is `Unsupported type passed: username`, if I remove username then it moves to the next field, `Unsupported type passed: imdb_title_id`. Here's the code I am using to run the query: ``` query = { TableName: 'RatingsTable', IndexName: 'username', KeyConditionExpression: '#username = :username', ExpressionAttributeNames: { '#username': 'username' }, ExpressionAttributeValues: { ':username': { S: username } }, ProjectionExpression: 'username, imdb_title_id' }; console.log('query', query); const result = await dbClient.send(new QueryCommand(query)); ``` Any ideas what I am doing wrong?

I would like to be able to query data hierarchically and return the results ordered by another attribute. What is the most efficient way to store and query sorted hierarchical data? For example, if I have a table with four attributes: `customer_id`, `country`, `location`, and `last_updated_date`, where `location` contains hierarchical information such as `state:county:city`, so a few records may look like: ``` ------------|--------|-------------------|-------------| customer_id |country |location |last_updated | ------------|--------|-------------------|-------------| 123456 |USA |WA:King:Seattle |2022-03-18 | 789012 |USA |WA:King:Kent |2022-03-15 | 098765 |USA |NY:Bronx:NYC |2022-02-28 | 432109 |USA |WA:Spokane:Spokane |2022-03-20 | ``` The `PK` of the table is the `customer_id` because most queries will pull information by `customer_id`, but there are other use cases that will want to (a) find all customers within a given location (e.g. `state` or `county`), and (b) return the results sorted (descending) by `last_updated`. To accomplish (a), I have a `GSI`, with `country` as the `PK` and `location` as the `SK`, `query`ing the `GSI` using `location.begins_with`. But I can't figure out how to accomplish (b). My understanding is that ordering operations are usually performed with `scanIndexForward`, but I'm already using the `GSI` for the hierarchical query. Is there a way to do both (a) and (b)? Thanks!

My source db is DynamoDB. It has 26 tables. I am trying to build AWS QuickSight reports using AWS Athena that builds the specific views for said reports. however, I seem to only be able to select a single table in creating the Glue job despite being able to select all tables i need for the crawler of the entire DB from Dynamo. What is the simplest route to get a complete extract of all tables that is queryable in Athena. I dont want to connect the reports direct to dynamoDB as it s a production database and want to create some separation to avoid any performance degradation by a poor query etc. How to extract these tables?

I just experienced disturbing behavior and I wonder if anyone else saw anything similar and how to handle it? I'm updating a row in DynamoDb from node.js lambda and logging ALL_NEW result of the update operation. In my log, I can see the data I was trying to save but in the table itself, that data is missing. On the next run, everything went OK. This is very problematic from a data consistency perspective. Anyone has an idea where to look and if there is some setting or way to handle this kind of behavior?

My client has an architecture to trigger a lambda whenever the dynamo db table is updated. However, my client needs to perform a check on the specific attribute which changed. The lambda payload gives the new and old image. However the lambda should compare and check what changed between the old and new image in order to proceed to the next step. In this case, is there any way the lambda can know which specific attribute changed when an item is updated in a dynamo db ?

Hi there, according to the [documentation ](https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_ExecuteStatement.html#API_ExecuteStatement_RequestSyntax) LIMIT is an optional parameter for DynamoDB PartiQL . However when I attempt to use it in the console by entering a statement like `select * from "USER" limit 1` I receive an error "ValidationException: Unsupported clause: LIMIT " Would be good if the console supported this since the API does

My question about DynamoDB is about this: https://aws.amazon.com/blogs/database/choosing-the-right-dynamodb-partition-key/ Where it says: “DynamoDB splits partitions by sort key if the collection size grows bigger than 10 GB.” Does that mean it will use the sort key for partitioning? As in PK+SK will be used to calculate the hash? Or just that it will be sorted by SK, so once a partition fills, the next partition will pick up where the other one left off? But it’s still only the PK used for calculating the hash I’m wondering if the cardinality of my partition key is determined by PK or PK+SK. If it’s the latter, then an operation like “Delete all items where PK=X”, would not have as big of a hot key / hot partition concern, because the SK should distribute the operation across many partitions. But if all items with the same PK are bundled up together in one partition (or a couple) then it will definitely be a problem.

Can the SAM Lambda AthenaDynamoDBConnector (Athena DynamoDB connector) be available for CDK? Currently it can only be deployed manually, is there a way to use CDK to deploy it?

While connecting to encrypted Amazon DAX Cluster using DAX Java SDK we are getting following error ``` failed to configure cluster endpoints from hosts: [<DAX-Cluster-Endpoint>:9111] ``` We are using following dax sdk maven dependency ``` `<dependencies> <dependency> <groupId>com.amazonaws</groupId> <artifactId>amazon-dax-client</artifactId> <version>1.0.230341.0</version> </dependency> </dependencies>` ``` Code Snippet ``` DynamoDB getDaxClient(String daxEndpoint) { AmazonDaxClientBuilder daxClientBuilder = AmazonDaxClientBuilder.standard(); daxClientBuilder.withRegion(region).withEndpointConfiguration(daxEndpoint); AmazonDynamoDB client = daxClientBuilder.build(); return new DynamoDB(client); } ``` DAX Cluster is in ap-south-1 region. However, I am able to connect DAX from local machine and ec2 instances when running nc command ``` nc -z <DAX-Cluster-Endpoint> 9111 Connection to<DAX-Cluster-Endpoint> port 9111 [tcp/*] succeeded! ``` Note : 9091 port is open for our VPC in security group associated with the DAX cluster

Over time AWS is doing a fantastic job of continually offering improved compute, memory and storage options and resources. However, the current service limits in DynamoDB, in particular the limit of 25 items per transaction is I would say very low. If you need good transactionality above this level you are limited in your options. For that reason I am proposing an increase to something in the realm of 100.

Hello, I have an application deployed in an EKS cluster in `Account A` that is trying to read an item from a DynamoDB table in `Account B`. I have done the following : * I have created a role in `Account B` called `DynDBReadAccess` with a policy that allows someone to perform the `dynamodb:GetItem` action on the table `arn:aws:dynamodb:us-east-1:<Account B>:table/myTable`. * I then created the role `CrossAccountDynDBAccess` in `Account A` with permissions to perform the `sts:AssumeRole` action and assume the role `arn:aws:iam::<Account B>:role/DynDBReadAccess`. * I updated the trust policy in the `DynDBReadAccess` role to trust the Principal `arn:aws:iam::<Account A>:role/CrossAccountDynDBAccess`. I have a pod with aws cli deployed in the cluster for debugging purposes. Now, I do the following. * I exec into the pod and run `aws sts get-caller-identity`, I see the correct assumed role `CrossAccountDynDBAccess`. * Then I run `aws sts assume-role --role-arn "arn:aws:iam::<Account B>:role/DynDBReadAccess" --role-session-name crossAccountSession` and I get the temporary credentials. * I set the environment variables `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` and `AWS_SESSION_TOKEN` with the temporary credentials I just received. * I run the following command `aws dynamodb get-item --table-name myTable --key 'somekey'` and I get the following error `An error occurred (AccessDeniedException) when calling the GetItem operation: User: arn:aws:sts::<Account B>:assumed-role/DynDBReadAccess/crossAccountSession is not authorized to perform: dynamodb:GetItem on resource: arn:aws:dynamodb:ap-southeast-1:<Account B>:table/myTable` I thought that once the roles, permissions and trust policies were set, cross account access should be possible. **Can someone tell me what is missing?** Some other points to note * The OIDC endpoint and IRSA role has been enabled in the EKS cluster and the service account for the cluster has been created. * The aws cli pod that I deployed has been deployed with the service account mapped to the IRSA role. * I tried doing the same thing with a lambda function where I created a lambda function in `Account A` that will read an item from the same DynamoDB table. The execution role of the lambda function assumes the `arn:aws:iam::<Account B>:role/DynDBReadAccess` and reads an item from the same DynamoDB table. **This works**.

The NoSQL Workbench software seems to require specifying an attribute _type_, even for attributes that are not part of any keys. In DynamoDB, however, I can have distinct items with the same attribute (named, e.g. "myAttr"), where for item A, myAttr might be a String, and for item B, myAttr might be a Map. Trying to use myAttr as a key attribute will then fail, but otherwise this appears to work fine in DynamoDB. Is there any way in the NoSQL Workbench to leave the type unspecified for some attributes? The UI doesn't seem to permit this :-/

Currently you get the error ValidationException: Only Select from a Single Table or index supported Are the plans to support joins/multi-select even if it is inefficient scans?

Hi, I have a use case where I am fetching data on certain items (unique itemID) multiple times a day (identified by day_BatchTime) and storing them on a dyanmoDB. My composite primary key consists of itemID & day_BatchTime. I have set itemID as parition key and day_BatchTime as sort key. But I need to report data for each day on a daily basis. So I tried setting up a global secondary index as feedDate. But running the query on global secondary index did not return any result (screenshot: https://ibb.co/kJ3p5mZ ). Please let me know if I am missing something and if there is a better way to achieve my objective of day wise items. I am using AWS console to fetch this data for now. Regards, Dbeings

Can you point me to someone that can confirm for a customer that NoSQL Workbench is not using Log4j?

Dear Experts, I have a long-running process that connects to DynamoDB using the C++ SDK. I have Route 53 health checks enabled, to which this process responds. If the process crashes, or the whole server goes down, Route 53 notices and fails over to a backup. However if the process' connection to DynamoDB fails then the health checker doesn't currently notice this because the health check response doesn't exercise that code. This has recently caused some downtime. I can of course modify the process to do a dummy DynamoDB action of some sort before responding to the health check. But I have a couple of reservations about that: - That's a lot of extra DynamoDB requests. I was wondering if there is some sort of simpler "ping" that I can send to DynamoDB. Or if there is a more generic service health ping common to all AWS services. - Alternatively, I could track the success/failure of the most recent non-health-check request and report that for the health check - but in that case, once Route 53 has failed over there are no more non-health-check requests, and it will be stuck in the failed state. It won't be difficult for me to hack something together, but the danger is constantly adding code that detects the last failure but won't detect the next one. I am posting this to ask if anyone has any best practice advice for how to do this "properly". Thanks, Phil.

I am using v3 of the aws-sdk. When I try to use the DynamoDB document client to put items into my DynamoDB table, I receive the error `Number 1213213131312312312314342 is greater than Number.MAX_SAFE_INTEGER. Use BigInt. ` (full stack trace below). My code is as follows: ``` const dynamodb = new DynamoDBClient({ region: 'us-east-1' }); const client = DynamoDBDocumentClient.from(dynamodb, translateConfig); const params = { TableName: 'Orders', Item: { PK: `${PK}`, SK: 'SK', id: id } const putOperation = new PutCommand(params); await client.send(putOperation) ``` Id is a BigInt. I was under the impression that the DocumentDB client would marshall the number to a 'BigInt', however it looks like it is only being marshalled to a 'Number' and since it is too large to be held by a Number it is failing the validation and throwing an exception. I also tried playing around with marshall/unmarshall configurations but saw the same error. Please let me know if you know how I can fix this issue/if it is a bug in the library. Additional information that might be helpful: Lambda Node runtime version specified in CDK ``` runtime: Runtime.NODEJS_14_X, ``` Library versions (all sdk v3) ``` "@aws-sdk/client-dynamodb": "^3.53.0", "@aws-sdk/lib-dynamodb": "^3.53.0", "@aws-sdk/util-dynamodb": "^3.53.0", ``` Full stacktrace. ``` 2022-03-09T20:11:12.601Z 9e321b10-3eb2-4042-b429-a8205e12f5b5 INFO Error: Number 820982911946154500 is greater than Number.MAX_SAFE_INTEGER. Use BigInt. at validateBigIntAndThrow (/var/task/node_modules/@aws-sdk/util-dynamodb/dist-cjs/convertToAttr.js:129:11) at convertToNumberAttr (/var/task/node_modules/@aws-sdk/util-dynamodb/dist-cjs/convertToAttr.js:138:9) at Object.convertToAttr (/var/task/node_modules/@aws-sdk/util-dynamodb/dist-cjs/convertToAttr.js:35:16) at /var/task/node_modules/@aws-sdk/util-dynamodb/dist-cjs/convertToAttr.js:118:36 at convertToMapAttrFromEnumerableProps (/var/task/node_modules/@aws-sdk/util-dynamodb/dist-cjs/convertToAttr.js:122:7) at Object.convertToAttr (/var/task/node_modules/@aws-sdk/util-dynamodb/dist-cjs/convertToAttr.js:23:16) at /var/task/node_modules/@aws-sdk/util-dynamodb/dist-cjs/convertToAttr.js:118:36 at convertToMapAttrFromEnumerableProps (/var/task/node_modules/@aws-sdk/util-dynamodb/dist-cjs/convertToAttr.js:122:7) at Object.convertToAttr (/var/task/node_modules/@aws-sdk/util-dynamodb/dist-cjs/convertToAttr.js:23:16) at Object.marshall (/var/task/node_modules/@aws-sdk/util-dynamodb/dist-cjs/marshall.js:5:53) ```

Hi, I am writing to dynamoDB using AWS lambda. I am using AWS Console to read data from dynamoDB. But I have seen instances of stale read with latest not records getting updated when trying to pull records under few mins. What is a safe time interval for data pull which would ensure that latest data is available on read? Would 30 mins be a safe interval? The below is from AWS site: Just want to understand how recent is recent here. "When you read data from a DynamoDB table, the response might not reflect the results of a recently completed write operation. The response might include some stale data" Regards, Dbeings

I send data to front-end client by APIGateway WebSocket with a Lambda in JS, triggered by a DynamoDB Stream : The JS Code to POST a message to the front-end is : ``` const params = { ConnectionId: clientId, Data: JSON.stringify(message) // message envoyé au client }; await client.postToConnection(params).promise() ``` When I use this code in a Lambda which is used by some routes in APIGateway it works perfectly. When I use an other lambda which is activate by a dynamoDB Stream, the lambda is executed when I insert new data and the post event appears on CloudWatch for the lambda but don't appears on CloudWatch for the APIGateway Socket and of course on the front-end ??? If I insert again new different data in dynamoDB, CloudWatch shows the Post of the first inserted data and API Gateway send it twice ??? And again and again there is always an offset in the message reception in the front-end. I can't understand why ???

I'm getting an error in CloudWatch that is: ConsumedWriteCapacityUnits The threshold says: ``` ConsumedWriteCapacityUnits < 30 for 15 datapoints within 15 minutes ``` I'm new to DynamoDB. My website looks up content in this table. It is very small, both the website and the table, so it is hard to believe I'm over doing anything? I'm guessing I need to make more "WriteCapacityUnits"? But I'm not sure how to do that? Any tips or advice is welcome!

Hello I am working on serverless application, and i was looking for something handle the frontend part and I found Honeycode since its native and its codeless. So is it possible to my Honeycode app save the data directly to Dynamodb so I can expose the read operation to those data via (API Gateway > lambda > Dynamodb) ?? one way i found is to use zapier to trigger Honeycode and insert in dynamodb, but i think we may use something native aws So my admin staff will use Honeycode to add products for example and my customers will expose the API

I'm new to DynamoDB and NoSQL, so I'm trying to get my head wrapped around table design. I have a simple app that will accept status reports about a series of locations. Each status report will have a location ID (number), the status (number), and a timestamp (string). I also need to store a description of each location, which will have a location ID (number) and four other values (all strings). The read pattern is to query the most recent status report for a subset of location IDs (that may change each time). I can use the design pattern where you query with a limit of 1 and set Scan Forward to false for each location ID. The report that will be built needs to include the status, the timestamp, and the description for each of the queried locations. The first part of the design is easy -- the partition key is the location ID, and the sort key is the timestamp. Next, though -- how do I store the description data? (Important: the app is supposed to be run via several Lambda functions, so I'm a bit limited in terms of caching the description info. Eventually, the number of locations will be in the thousands.) Option 1: Store the description info in each status report. I will need to do two transactions each time a status report comes in -- read the description info, then write the new status report item including the four description fields. Reads then become easy, as the info is all there. Option 2: Store the store the descriptions as separate items in the same table. Use a Global Secondary Index to query the description info. When a report is run, do two reads for each location, one to get the status and timestamp, one to get the description. Receiving a status report needs only one transaction, a write. If this is the best option, how should I structure the GSI? Option 3: Store the descriptions in a separate table. Everything else is as per Option 2. Any and all advice greatly appreciated.

Dear Experts, DynamoDB GetItem operations can be read-consistent or not. But there isn't a corresponding setting for DeleteItem. So are DeleteItem operations consistent? I.e. if one connection does a PutItem, and then another connection does a DeleteItem with the same key, is it guaranteed that the just-put item is actually deleted? In my case I'm using SetReturnValues(ALL_OLD) to both read and delete an item, if that makes any difference. Thanks, Phil.

I met a really weird question when I was using DynamoDB in the project (with JAVA SDK). We created a DynamoDB table with a Primary Partition Key 'id', and a Sort Key 'create_time'. And I used `query` operation in my source code with both Partition Key and Sort Key, but I get a exception while querying ``` com.amazonaws.services.dynamodbv2.model.AmazonDynamoDBException: Query condition missed key schema element: create_time (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: ValidationException; Request ID: K9B8K5DU0CQ1BQA9VJIPTI6EIJVV4KQNSO5AEMVJF66Q9ASUAAJG) ``` I have researched for a day, and almost all the results for searching "Query condition missed key schema element" from google told me that I'm not using the Partition Key or GSI in `KeyConditionExpression` , but I'm sure that we had set up correct Partition Key and Sort key with correct names and used correct in source code, by the way we didn't create any GSI in out DynamoDB. So I just want to know that are there any situations which might cause this exception - Query condition missed key schema element with a **Sort Key** rather than a **Partition Key**. **BACKGROUND STATEMENT ** 1. This `query` operation worked in my develop environment well, but not worked in the production environment. The source code is exactly the same. 2. This table is new created in production environment. (I ever suspected that there must be something wrong while creating the table in the production environment. But I have confirmed for several times, the Partition Key and Sort Key are all correctly named and we do not create any GSI.) 3. The differences between develop and production might be the DynamoDB regions. Develop is created in ap-southeast-1 and the production is in us-west-1.

**It is possible to consume secretsManager via dynamoDb tables? for example : ** ``` { key: {{resolve:secretsmanager::password}} } ```

Greetings, We are using AWS IoT Core services for collecting data from our telematics device. The data is further stored into the DynamoDb database via a rule engine. On publishing the shadow document to the broker the one value field (uint64_t) gets changed on the . The value retains its original value in the dynamoDb. Device Shadow Expected ``` { "state": { "reported": { "vsr": 70931715605954314 } } } ``` Device Shadow Actual ``` { "state": { "reported": { "vsr": 70931715605954310 } } } ``` Is there any chance there is some data conversion issue on AWS IoT Core side. I have a thingsboard client connected to the endpoint as well it is also receiving the manipulated device shadow Regards, Altamash