Transit Gateway Logs - Bug report

0

I created an Athena table for vpc flow logs according to the documentation. However I noticed an inconsistency when comparing TG flow logs to other VPC flow logs.

For other VPC flow logs the column account_id contains my account id (int) and interface_id contains eg 'eni-0eb0ac3356dee8030'.
For TG logs account_id contains the string 'TransitGateway' and the interface_id contains my account id.

To me this seems like a bug. Especially since it will mess up all tables that expect an int for account_id...

질문됨 2년 전388회 조회
2개 답변
1

This can be done using the below query:

CREATE EXTERNAL TABLE IF NOT EXISTS tgwflowlogspqttbl ( version int, resource_type string, account_id string, tgw_id string, tgw_attachment_id string, tgw_src_vpc_account_id string, tgw_dst_vpc_account_id string, tgw_src_vpc_id string, tgw_dst_vpc_id string, tgw_src_subnet_id string, tgw_dst_subnet_id string, tgw_src_eni string, tgw_dst_eni string, tgw_src_az_id string, tgw_dst_az_id string, tgw_pair_attachment_id string, srcaddr string, dstaddr string, srcpor int, dstport int, protocol bigint, packet bigint, bytes bigint, start bigint, end bigint, log_status string, type string, packets_lost_no_route bigint, packets_lost_blackhole bigint, packets_lost_mtu_exceeded bigint, packets_lost_ttl_expired bigint, tcp_flags int, region string, flow_direction string, pkt_src_aws_service string, pkt_dst_aws_service string) ROW FORMAT SERDE 'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe' STORED AS INPUTFORMAT 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat' LOCATION 's3://bucket/tgw-logs-parquet/' TBLPROPERTIES ( 'skip.header.line.count'='1' )

AWS
답변함 5달 전
0

Hi There

Querying TGW flow logs with Athena is not currently supported

See https://docs.aws.amazon.com/vpc/latest/tgw/tgw-flow-logs.html

You can search TGW flow logs using Cloudwatch

See https://docs.aws.amazon.com/vpc/latest/tgw/working-with-flow-logs.html#search-flow-log-records

profile pictureAWS
전문가
Matt-B
답변함 2년 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인