How do I see a list of my Amazon EC2 instances that are connected to Amazon EFS?

Short description

To track traffic for each Amazon EFS mount target's network interface, use Amazon Virtual Private Cloud (Amazon VPC) Flow Logs. Publish the flow logs to Amazon CloudWatch Logs. Then, use CloudWatch Logs Insights to filter the traffic flow on the mount target's network interface. CloudWatch Logs Insights uses a specific timestamp to identify EC2 instances with mounted EFS file systems.

Resolution

Note: If you receive errors when you run AWS Command Line Interface (AWS CLI) commands, then see Troubleshooting errors for the AWS CLI. Also, make sure that you're using the most recent AWS CLI version.

Create a log group

Complete the following steps:

1. Open the Amazon CloudWatch console.
2. In the navigation pane, choose Logs, and then choose Log groups.
3. Choose Create log group.
   For Log group name, enter a name.
   For Retention setting, enter a retention setting.
   (Optional) Add an AWS Key Management Service (AWS KMS) key Amazon Resource Name (ARN) and tags.
4. Choose Create.

Create an IAM role with permissions to publish flow logs to CloudWatch Logs

Complete the following steps:

1. Open the AWS Identity and Access Management (IAM) console.
2. Under Access management, choose Roles.
3. Choose Create role, and then create a new IAM role.
4. In the role's IAM policy, add the permissions and include a trust policy for the service to assume the role.

Get your Amazon EFS file system's mount target network interfaces

Note: EFS has a different mount target for each Availability Zone.

Complete the following steps:

1. Open the EFS console.
2. Under File systems, choose your EFS file system, and then choose View details.
3. Choose Network, and then note the network interface ID for each mount target.

Create the flow logs

Complete the following steps:

1. Open the EC2 console.
2. Choose Network & Security, and then choose Network Interfaces.
3. Choose the network interfaces that the mount target uses.
4. On the Actions menu, choose Create flow log.
5. Enter the following values:
   (Optional) For Name, enter a flow log name.
   For Filter, choose All.
   For Maximum aggregation interval, choose either the default 10 minutes or 1 minute.
   For Destination, choose Send to CloudWatch logs.
   For Destination log group, choose your log group.
   For IAM role, choose your IAM role.
   For Log record format, choose either AWS default format or Custom format.
   (Optional) For Tags, add tags.
6. Choose Create.
7. To monitor the flow log status, choose your network interface, and then choose Flow logs. Verify that the Status is Active.

Note: The flow logs system publishes the first log to CloudWatch Logs after approximately 10 minutes.

Verify that the flow logs are in CloudWatch Logs

Complete the following steps:

1. Open the CloudWatch console.
2. In the navigation pane, choose Log groups.
3. For Log groups, choose your log group.
4. Verify that all the log streams appear.
   Note: Each network interface has a different log stream.

Use CloudWatch Logs Insights to run a query

Complete the following steps:

1. Open the CloudWatch console.
2. In the navigation pane, choose Logs, and then choose Logs Insights.
3. Choose your log group.
4. Choose one of the following duration values: 5m, 30m, 1h, 3h, 12h, or Custom.
5. Run the following query:

   fields @timestamp, @message | filter dstPort="2049" | stats count(*) as FlowLogEntries by srcAddr | sort FlowLogEntries desc

The preceding query reviews all flow logs for all mount targets. The query filters the logs that have a destination port set to Port=2049. EFS clients connect to mount targets on NFS port 2049. The query retrieves all unique EFS client IP addresses and sorts them by the most active client connections. The number of entries in the flow log determines the activity level.

The query output contains the list of private IP addresses for all instances with mounted EFS file systems.

Example query output:

#          srcAddr              FlowLogEntries1      111.22.33.44                 782      111.55.66.77                36
3      111.88.99.000                33

Run the start-query command

Check that jq is installed:

yum install -y jq

Then, run the following start-query command:

aws logs start-query --log-group-name EFS-ENI-Flowlogs --start-time 1643127618 --end-time 1643128901 --query-string 'filter dstPort="2049" | stats count(*) as FlowLogEntries by srcAddr | sort FlowLogEntries desc' > test.json && sleep 10 && jq .queryId test.json | xargs aws logs get-query-results --query-id

In the command, set the following parameters:

• For log-group-name, enter your log group name.
• For start-time and end-time, enter your start and end times.
  Note: These values are in Unix Epoch time. To convert to readable timestamps, use the Epoch & Unix timestamp conversion tools on the Epoch converter website.
• (Optional) For test.json, you can change the JSON file name each time you run the command. When you change the name, the previous output doesn't merge with the new output.
• For sleep, to review the flow logs when the query is running, specify the delay time in seconds.

To view the IP addresses of clients that currently mount EFS file systems, run a new query.