Sending Metrics from AWS Firehose to NewRelic metrics api

Question

I follow NewRelic steps to integrate my AWS account, i used the provided CloudFormation stack so everything configured as expected.

I am trying to send metrics from AWS account to https://aws-api.newrelic.com/cloudwatch-metrics/v1 using AWS firehose, I believe i configured everything correctly, Firehose using Secret Manager (I know its working as when i change the secret i get another error):

My current Secret JSON format is like this:

{"api_key": "NRAK-"}

I get the following error in Firehose:
{"attemptsMade":6,"arrivalTimestamp":,"errorCode":"HttpEndpointDestinationException","errorMessage":"403: {\"errorMessage\":\"Error ingesting request\",\"reques

I even tried to configured the API Key direct in Firehose, instead of using Secrets Manager.
I can access the api endpoint through curl using the API KEY which proving the key is correct.

re:Post Agent · Answer

Based on the error you're seeing, it appears that your Firehose delivery stream is successfully connecting to the New Relic endpoint but is receiving a 403 Forbidden response. This indicates a permissions issue rather than a connectivity problem.

The "HttpEndpointDestinationException" with a 403 status code suggests that the access key you've configured for your Firehose stream doesn't have sufficient permissions to deliver data to the New Relic endpoint. Even though you can access the API endpoint through curl using the same API key, there might be differences in how Firehose is formatting or sending the request.

Here are some troubleshooting steps to consider:

1. Verify the format of your API key in the Secret Manager. The format you provided (`{"api_key": "NRAK-"}`) looks correct, but ensure the full API key is included and properly formatted.

2. Check if the New Relic API key has the correct permissions to ingest metrics data. Some API keys might be read-only or have limited scopes.

3. Review the content encoding and format settings in your Firehose configuration to ensure they match what New Relic expects.

4. If you're using a Lambda function for data transformation before delivery, verify that it's formatting the data correctly for New Relic's API.

5. Check if there are any specific headers or request parameters that New Relic requires that might not be included in your Firehose configuration.

Since you mentioned that changing the secret results in a different error, the connection to Secret Manager is working correctly. The issue appears to be with the authorization or format of the data being sent to New Relic.  
**Sources**  
[Troubleshooting HTTP Endpoints - Amazon Data Firehose](https://docs.aws.amazon.com/firehose/latest/dev/http_troubleshooting.html)

Sending Metrics from AWS Firehose to NewRelic metrics api

Add your answer

Relevant content