Transit Gateway Logs - Bug report

0

I created an Athena table for vpc flow logs according to the documentation. However I noticed an inconsistency when comparing TG flow logs to other VPC flow logs.

For other VPC flow logs the column account_id contains my account id (int) and interface_id contains eg 'eni-0eb0ac3356dee8030'.
For TG logs account_id contains the string 'TransitGateway' and the interface_id contains my account id.

To me this seems like a bug. Especially since it will mess up all tables that expect an int for account_id...

已提问 2 年前305 查看次数
2 回答
1

This can be done using the below query:

CREATE EXTERNAL TABLE IF NOT EXISTS tgwflowlogspqttbl ( version int, resource_type string, account_id string, tgw_id string, tgw_attachment_id string, tgw_src_vpc_account_id string, tgw_dst_vpc_account_id string, tgw_src_vpc_id string, tgw_dst_vpc_id string, tgw_src_subnet_id string, tgw_dst_subnet_id string, tgw_src_eni string, tgw_dst_eni string, tgw_src_az_id string, tgw_dst_az_id string, tgw_pair_attachment_id string, srcaddr string, dstaddr string, srcpor int, dstport int, protocol bigint, packet bigint, bytes bigint, start bigint, end bigint, log_status string, type string, packets_lost_no_route bigint, packets_lost_blackhole bigint, packets_lost_mtu_exceeded bigint, packets_lost_ttl_expired bigint, tcp_flags int, region string, flow_direction string, pkt_src_aws_service string, pkt_dst_aws_service string) ROW FORMAT SERDE 'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe' STORED AS INPUTFORMAT 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat' LOCATION 's3://bucket/tgw-logs-parquet/' TBLPROPERTIES ( 'skip.header.line.count'='1' )

AWS
已回答 23 天前
0

Hi There

Querying TGW flow logs with Athena is not currently supported

See https://docs.aws.amazon.com/vpc/latest/tgw/tgw-flow-logs.html

You can search TGW flow logs using Cloudwatch

See https://docs.aws.amazon.com/vpc/latest/tgw/working-with-flow-logs.html#search-flow-log-records

profile pictureAWS
专家
Matt-B
已回答 2 年前

您未登录。 登录 发布回答。

一个好的回答可以清楚地解答问题和提供建设性反馈,并能促进提问者的职业发展。

回答问题的准则