Using Amazon Athena to read data captured by the AWS Application Discovery Agent

3 minute read
Content level: Advanced
0

Demonstrates enabling Athena export from Migration Hub, and retrieving data from the Discovery Agent database

Situation

A customer is using Amazon Q Developer transformation capabilities for VMware to build wave groups for a migration into Amazon EC2. They are using the AWS Application Discovery Agent to obtain network data for Amazon Q to analyze. Amazon Q suggested some unexpected wave groups, and the customer wants to view the underlying network data collected by the Application Discovery Service.

In the lab setup for this post, I created two Windows VMs, installed the Discovery Agent, then started a script that ran in a loop, continuously copying a file between them to generate network traffic.

Task

  1. Enable Amazon Athena exports in the Migration Hub
  2. Query the collector database

Action

A. Enable Amazon Athena exports

  1. In AWS Migration Hub, navigate to Data collectors, then Discovery Agents.
  2. There is a toggle switch labeled Athena export. Select the toggle to turn it on. It could take 20 minutes or longer to enable.
  3. Take note of the hostnames and Agent IDs that apply to the VMs that you are investigating.

Enable Athena

B. Launch Amazon Athena

  1. In AWS Migration Hub, navigate to Servers
  2. Filter the list by searching for a VM, select the VM, then select Actions, then View in Amazon Athena

C. Run queries in Athena

There are several tables available in the Application Discovery Server database. Here, we focus on outbound_connection_agent, which stores all outbound connections from a VM with an agent installed.

outbound columns

Using the agent ID noted above, I construct a query to look for Windows file copying activity, which takes place over TCP/445

SELECT * FROM outbound_connection_agent WHERE agent_id='o-123abc456' AND destination_port=445

Athena query

The first row shows file copy from my source VM to my domain controller, which makes sense for SYSVOL and other related traffic that is copied over TCP/445. The second row shows the expected file copy activity from my source VM to my destination VM. This proves that communication in my lab is being detected correctly by the agent.

File copy activity

Cleanup

  1. In AWS Migration Hub, navigate to Data collectors, then Discovery Agents.
  2. There is a toggle switch labeled Athena export. Select the toggle to turn it off.

Result

The customer was able to validate network communication using data in the tables.

For more information, visit the Exploring data in Amazon Athena page in the user guide.