How to pull a list of items from a DynomoDB table without using primary key in Go.

Question

Below is a single item in JSON, there are hundreds more.  The account_name and ami_name are the partition key and sort key.  I want to be able to query a list based on the ami_name and filter that list by the environment value, in this case, "test".  I have an example of what I am trying so far.
```

package main

import (
    "context"
    "fmt"
	"log"
    "github.com/aws/aws-sdk-go-v2/aws"
    "github.com/aws/aws-sdk-go-v2/config"
    "github.com/aws/aws-sdk-go-v2/service/dynamodb"
    "github.com/aws/aws-sdk-go-v2/service/dynamodb/types"
)

func main() {
    cfg, err := config.LoadDefaultConfig(context.TODO(), config.WithRegion("us-east-1"))
    if err != nil {
        log.Fatalf("unable to load SDK config, %v", err)
    }

tableName   := "aws-webhooks-table-test"
	// account_name := "aws-computer-test"
	ami_name     := "win2019core"
    environment := "test"
    svc := dynamodb.NewFromConfig(cfg)
    out, err := svc.Query(context.TODO(), &dynamodb.QueryInput{
        TableName:              aws.String(tableName),
        KeyConditionExpression: aws.String("ami_name = :ami_name"),
		FilterExpression: aws.String("environment = :environment"),
        ExpressionAttributeValues: map[string]types.AttributeValue{
            ":ami_name":  &types.AttributeValueMemberS{Value: ami_name},
			":environment":  &types.AttributeValueMemberS{Value: environment},
        },
        ExpressionAttributeNames: map[string]string{
            "ami_name": ami_name,
        },
}

{
  "account_name": {
    "S": "aws-computer-test"
  },
  "ami_name": {
    "S": "win2019core"
  },
  "account_id": {
    "S": "7666666666"
  },
  "aws_region": {
    "S": "us-east-1"
  },
  "environment": {
    "S": "test"
  },
  "lob": {
    "S": "computer"
  },
  "payload_url": {
    "S": "https://codebuild.us-east-1.amazonaws.com/webhooks?t=eyJlbmNyeXBlotsoflettersKPIkeo"
  },
  "project_name": {
    "S": "aws-computer-test-win2019-ami-project"
  },
  "repo_url": {
    "S": "https://git.nylcloud.com/Cloud-Team/packer-aws-win2019core"
  },
  "secret": {
    "S": "dq7bHlotsofletterZKPIkeo"
  }
}
```

Answer

In DynamoDB you must always provide the Partition Key when reading items, unless using `Scan` operation which will read every item in the table and is best to be avoided if possible.

For your use-case you would need to create a Global Secondary Index (GSI) and specify `ami_name` as the partition key. Optionally you can use `environment` as your sort key.

Now you would target the GSI and use a `Query` operation to read the data (with partition key only):

```
    out, err := svc.Query(context.TODO(), &dynamodb.QueryInput{
        TableName:              aws.String(tableName),
        IndexName:              aws.String(indexName),
        KeyConditionExpression: aws.String("ami_name = :ami_name"),
	FilterExpression: aws.String("environment = :environment"),
        ExpressionAttributeValues: map[string]types.AttributeValue{
            ":ami_name":  &types.AttributeValueMemberS{Value: ami_name},
	    ":environment":  &types.AttributeValueMemberS{Value: environment},
        },
        ExpressionAttributeNames: map[string]string{
            "ami_name": ami_name,
        },
}
```

If you choose to use `environment` as sort-key on your index then you would include that in your `KeyConditionExpression` and remove the `FilterExpression`.

Answer

Yes you can use the pagination method which uses 'Scan' operation, however this operation will read every item in the table and is best to be avoided if possible due to two reasons :
1) Increase in cost.
2) Time consuming.

Therefore you must always provide the Partition Key when reading items.

How to pull a list of items from a DynomoDB table without using primary key in Go.

関連するコンテンツ