AWS file formats

Question

I am trying to construct the automated Detective Investigation using https://aws.amazon.com/blogs/security/how-to-use-the-amazon-detective-api-to-investigate-guardduty-security-findings-and-enrich-data-in-security-hub/.

It seems there are a few variants of JSON files that are used to communicate between services. Specifically, I am referring to GuardDuty Findings which show as one format but are reformatted to something slightly different somewhere along the way for the Lambda function, EventBridge, SecurityHub, and Detective.

I am having difficulty testing the code for the automated investigation because I cannot give it the file it will read. I have been unsuccessful using conversion with ChatGPT and I don't want to find a template that makes the code work but is not the actual format that will be read. The finding we can view by the link in GuardDuty is not ASFF.

How do I see the exact file that is being sent across each of these services? I want to see exactly what is being sent to SecurityHub, to EventBridge, to the Lambda, and to Detective. Also, I want a way to recreate any event in those formats using the sample events that GuardDuty can generate so I can test any service I design.

re:Post Agent · Answer

To understand and test the exact file formats being sent across different AWS services in the automated Detective investigation process, you'll need to consider a few key points:

1. GuardDuty to Security Hub:
GuardDuty sends its findings to Security Hub using the AWS Security Finding Format (ASFF). While the GuardDuty console may display findings in a different format, the integration with Security Hub automatically converts them to ASFF. You can view these ASFF-formatted findings directly in the Security Hub console.

2. Security Hub to EventBridge:
Security Hub sends findings to EventBridge in ASFF format. You can set up an EventBridge rule to capture these events for testing purposes.

3. EventBridge to Lambda:
The Lambda function receives the event from EventBridge, which is still in ASFF format. You can use the AWS Lambda console to test your function with sample ASFF-formatted events.

4. Lambda to Detective:
The Lambda function uses the Detective API to start an investigation. The input for this API call is derived from the ASFF-formatted finding, but it's not the entire ASFF finding itself.

To see and recreate these formats:

1. Use the Security Hub console to view ASFF-formatted findings.
2. Set up an EventBridge rule to capture Security Hub events and send them to a test Lambda function that logs the entire event.
3. Use the AWS CLI or SDKs to retrieve ASFF-formatted findings from Security Hub programmatically.
4. Use the sample events feature in the Lambda console to test your function with ASFF-formatted events.
5. For Detective API calls, refer to the AWS SDK documentation for the correct input format.

To recreate events using GuardDuty sample findings:
1. Generate sample findings in GuardDuty.
2. Wait for these findings to appear in Security Hub (this may take a few minutes).
3. Retrieve the ASFF-formatted findings from Security Hub.
4. Use these ASFF-formatted findings to test your Lambda function and other components of your automated investigation process.

By following these steps, you can ensure that you're working with the actual formats used in the production environment, allowing for more accurate testing and development of your automated investigation process.  
**Sources**  
[How to use the Amazon Detective API to investigate GuardDuty security findings and enrich data in Security Hub | AWS Security Blog](https://aws.amazon.com/blogs/security/how-to-use-the-amazon-detective-api-to-investigate-guardduty-security-findings-and-enrich-data-in-security-hub/)  
[Integrating with AWS Security Hub - Amazon GuardDuty](https://docs.aws.amazon.com/guardduty/latest/ug/securityhub-integration.html)

AWS file formats

Add your answer

Relevant content