How to pull a list of items from a DynomoDB table without using primary key in Go.

0

Below is a single item in JSON, there are hundreds more. The account_name and ami_name are the partition key and sort key. I want to be able to query a list based on the ami_name and filter that list by the environment value, in this case, "test". I have an example of what I am trying so far.



package main

import (
    "context"
    "fmt"
	"log"
    "github.com/aws/aws-sdk-go-v2/aws"
    "github.com/aws/aws-sdk-go-v2/config"
    "github.com/aws/aws-sdk-go-v2/service/dynamodb"
    "github.com/aws/aws-sdk-go-v2/service/dynamodb/types"
)

func main() {
    cfg, err := config.LoadDefaultConfig(context.TODO(), config.WithRegion("us-east-1"))
    if err != nil {
        log.Fatalf("unable to load SDK config, %v", err)
    }

	tableName   := "aws-webhooks-table-test"
	// account_name := "aws-computer-test"
	ami_name     := "win2019core"
    environment := "test"
    svc := dynamodb.NewFromConfig(cfg)
    out, err := svc.Query(context.TODO(), &dynamodb.QueryInput{
        TableName:              aws.String(tableName),
        KeyConditionExpression: aws.String("ami_name = :ami_name"),
		FilterExpression: aws.String("environment = :environment"),
        ExpressionAttributeValues: map[string]types.AttributeValue{
            ":ami_name":  &types.AttributeValueMemberS{Value: ami_name},
			":environment":  &types.AttributeValueMemberS{Value: environment},
        },
        ExpressionAttributeNames: map[string]string{
            "ami_name": ami_name,
        },
}

{
  "account_name": {
    "S": "aws-computer-test"
  },
  "ami_name": {
    "S": "win2019core"
  },
  "account_id": {
    "S": "7666666666"
  },
  "aws_region": {
    "S": "us-east-1"
  },
  "environment": {
    "S": "test"
  },
  "lob": {
    "S": "computer"
  },
  "payload_url": {
    "S": "https://codebuild.us-east-1.amazonaws.com/webhooks?t=eyJlbmNyeXBlotsoflettersKPIkeo"
  },
  "project_name": {
    "S": "aws-computer-test-win2019-ami-project"
  },
  "repo_url": {
    "S": "https://git.nylcloud.com/Cloud-Team/packer-aws-win2019core"
  },
  "secret": {
    "S": "dq7bHlotsofletterZKPIkeo"
  }
}
asked a month ago24 views
2 Answers
2

In DynamoDB you must always provide the Partition Key when reading items, unless using Scan operation which will read every item in the table and is best to be avoided if possible.

For your use-case you would need to create a Global Secondary Index (GSI) and specify ami_name as the partition key. Optionally you can use environment as your sort key.

Now you would target the GSI and use a Query operation to read the data (with partition key only):

    out, err := svc.Query(context.TODO(), &dynamodb.QueryInput{
        TableName:              aws.String(tableName),
        IndexName:              aws.String(indexName),
        KeyConditionExpression: aws.String("ami_name = :ami_name"),
	FilterExpression: aws.String("environment = :environment"),
        ExpressionAttributeValues: map[string]types.AttributeValue{
            ":ami_name":  &types.AttributeValueMemberS{Value: ami_name},
	    ":environment":  &types.AttributeValueMemberS{Value: environment},
        },
        ExpressionAttributeNames: map[string]string{
            "ami_name": ami_name,
        },
}

If you choose to use environment as sort-key on your index then you would include that in your KeyConditionExpression and remove the FilterExpression.

profile picture
answered a month ago
-2

Yes you can use the pagination method which uses 'Scan' operation, however this operation will read every item in the table and is best to be avoided if possible due to two reasons :

  1. Increase in cost.
  2. Time consuming.

Therefore you must always provide the Partition Key when reading items.

SUPPORT ENGINEER
answered a month ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions