Transit Gateway Logs - Bug report

0

I created an Athena table for vpc flow logs according to the documentation. However I noticed an inconsistency when comparing TG flow logs to other VPC flow logs.

For other VPC flow logs the column account_id contains my account id (int) and interface_id contains eg 'eni-0eb0ac3356dee8030'.
For TG logs account_id contains the string 'TransitGateway' and the interface_id contains my account id.

To me this seems like a bug. Especially since it will mess up all tables that expect an int for account_id...

已提問 2 年前檢視次數 306 次
2 個答案
1

This can be done using the below query:

CREATE EXTERNAL TABLE IF NOT EXISTS tgwflowlogspqttbl ( version int, resource_type string, account_id string, tgw_id string, tgw_attachment_id string, tgw_src_vpc_account_id string, tgw_dst_vpc_account_id string, tgw_src_vpc_id string, tgw_dst_vpc_id string, tgw_src_subnet_id string, tgw_dst_subnet_id string, tgw_src_eni string, tgw_dst_eni string, tgw_src_az_id string, tgw_dst_az_id string, tgw_pair_attachment_id string, srcaddr string, dstaddr string, srcpor int, dstport int, protocol bigint, packet bigint, bytes bigint, start bigint, end bigint, log_status string, type string, packets_lost_no_route bigint, packets_lost_blackhole bigint, packets_lost_mtu_exceeded bigint, packets_lost_ttl_expired bigint, tcp_flags int, region string, flow_direction string, pkt_src_aws_service string, pkt_dst_aws_service string) ROW FORMAT SERDE 'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe' STORED AS INPUTFORMAT 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat' LOCATION 's3://bucket/tgw-logs-parquet/' TBLPROPERTIES ( 'skip.header.line.count'='1' )

AWS
已回答 25 天前
0

Hi There

Querying TGW flow logs with Athena is not currently supported

See https://docs.aws.amazon.com/vpc/latest/tgw/tgw-flow-logs.html

You can search TGW flow logs using Cloudwatch

See https://docs.aws.amazon.com/vpc/latest/tgw/working-with-flow-logs.html#search-flow-log-records

profile pictureAWS
專家
Matt-B
已回答 2 年前

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南