Failing to access Neptune endpoints using (aws)curl

0

Hi, I'm trying to query the SPARQL endpoint of Neptune v. 1.1.0.0. I am trying to use IAM auth with temp credentials (for a Neptune-specific IAM user) and I have Neptune configured with IAM db authentication: Enabled

This is my setup script that I run from EC2 (Ubuntu) that is in a public subnet in the same VPC as Neptune:

tmpfile=$(mktemp)

unset AWS_ACCESS_KEY_ID
unset AWS_SECRET_ACCESS_KEY
unset AWS_SESSION_TOKEN

echo "AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID"
echo "AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY"
echo "AWS_SESSION_TOKEN: $AWS_SESSION_TOKEN"

aws sts assume-role \
        --profile default \
        --duration-seconds 1800 \
        --role-arn "arn:aws:iam::589606889449:role/NeptuneClient" \
        --role-session-name AWSCLI-Session > "$tmpfile"

AccessKeyId=$(cat "$tmpfile" | jq -r '.Credentials''.AccessKeyId')
SecretAccessKey=$(cat "$tmpfile" | jq -r '.Credentials''.SecretAccessKey')
SessionToken=$(cat "$tmpfile" | jq -r '.Credentials''.SessionToken')

export AWS_ACCESS_KEY_ID="$AccessKeyId"
export AWS_SECRET_ACCESS_KEY="$SecretAccessKey"
export AWS_SESSION_TOKEN="$SessionToken"

echo "AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID"
echo "AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY"
echo "AWS_SESSION_TOKEN: $AWS_SESSION_TOKEN"

aws sts get-caller-identity

get-caller-identity shows "Arn": "arn:aws:sts::589606889449:assumed-role/NeptuneClient/AWSCLI-Session" as expected.

The status endpoint works as well:

awscurl 'https://xxxxxxxxxxxxcluster.cluster-ro-cnol6sn9sq5j.us-east-1.neptune.amazonaws.com:8182/status' \
    --region us-east-1 \
    --service neptune-db

I get {"status":"healthy" ...} etc.

I was following the examples from Using RDF and SPARQL to access the graph in Amazon Neptune and replacing curl with awscurl. However if try to execute a query on a SPARQL endpoint, awscurl fails:

awscurl 'https://xxxxxxxxxxxxcluster.cluster-ro-cnol6sn9sq5j.us-east-1.neptune.amazonaws.com:8182/sparql' \
    -X POST \
    --data-binary \
    -d 'query=ASK {}' \
    --region us-east-1 \
    --service neptune-db

with this error:

Traceback (most recent call last):
  File "/home/ubuntu/.local/bin/awscurl", line 8, in <module>
    sys.exit(main())
  File "/home/ubuntu/.local/lib/python3.10/site-packages/awscurl/awscurl.py", line 543, in main
    inner_main(sys.argv[1:])
  File "/home/ubuntu/.local/lib/python3.10/site-packages/awscurl/awscurl.py", line 516, in inner_main
    response = make_request(args.request,
  File "/home/ubuntu/.local/lib/python3.10/site-packages/awscurl/awscurl.py", line 107, in make_request
    canonical_request, payload_hash, signed_headers = task_1_create_a_canonical_request(
  File "/home/ubuntu/.local/lib/python3.10/site-packages/awscurl/awscurl.py", line 219, in task_1_create_a_canonical_request
    payload_hash = sha256_hash_for_binary_data(data) if data_binary else sha256_hash(data)
  File "/home/ubuntu/.local/lib/python3.10/site-packages/awscurl/utils.py", line 20, in sha256_hash_for_binary_data
    return hashlib.sha256(val).hexdigest()
TypeError: Strings must be encoded before hashing

If I remove the --data-binary arg

awscurl 'https://xxxxxxxxxxxxcluster.cluster-ro-cnol6sn9sq5j.us-east-1.neptune.amazonaws.com:8182/sparql' \
        -X POST \
        -d 'query=ASK {}' \
        --region us-east-1 \
        --service neptune-db

I get a Neptune error:

{"requestId":"2ec4e671-f734-5519-03fd-6971f5229051","detailedMessage":"Missing 'query' or 'update' parameter for POST request","code":"MissingParameterException"}

In normal curl the argument would be --data-binary 'query=ASK {}', but apparently not in awscurl because this this form fails too:

usage: awscurl [-h] [-v] [-i] [-X REQUEST] [-d DATA] [-H HEADER] [-k] [--data-binary] [--region REGION] [--profile PROFILE] [--service SERVICE] [--access_key ACCESS_KEY] [--secret_key SECRET_KEY] [--security_token SECURITY_TOKEN]
               [--session_token SESSION_TOKEN] [-L]
               uri
awscurl: error: unrecognized arguments:  query=ASK {}

Eventually I tried accessing the status endpoint using plain curl (version 7.81.0 (x86_64-pc-linux-gnu)) as described in this blog post:

curl 'https://xxxxxxxxxxxxcluster.cluster-ro-cnol6sn9sq5j.us-east-1.neptune.amazonaws.com:8182/status' -v \
    --aws-sigv4 "aws:amz:us-east-1:neptune-db" \
    --user "${AWS_ACCESS_KEY_ID}:${AWS_SECRET_ACCESS_KEY}" \
    --header "x-amz-security-token: ${AWS_SESSION_TOKEN}" \
    --no-progress-meter

but this failed with yet another error:

{"requestId":"1ec4e65d-540b-93b3-a41f-9bd06f605385","detailedMessage":"The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.","code":"AccessDeniedException"}

This left me unsatisfied and exhausted :) Can someone please tell me what's going wrong here?

gefragt vor 9 Monaten468 Aufrufe
1 Antwort
0

This worked!

awscurl 'https://xxxxxxxxxxxxcluster.cluster-ro-cnol6sn9sq5j.us-east-1.neptune.amazonaws.com:8182/sparql' \
        -X POST \
        -d 'query=ASK {}' \
        --region us-east-1 \
        --service neptune-db \
        --header 'Content-Type: application/x-www-form-urlencoded'

Setting --header is currently an awscurl workaround.

beantwortet vor 9 Monaten

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen