Transit Gateway Logs - Bug report

0

I created an Athena table for vpc flow logs according to the documentation. However I noticed an inconsistency when comparing TG flow logs to other VPC flow logs.

For other VPC flow logs the column account_id contains my account id (int) and interface_id contains eg 'eni-0eb0ac3356dee8030'.
For TG logs account_id contains the string 'TransitGateway' and the interface_id contains my account id.

To me this seems like a bug. Especially since it will mess up all tables that expect an int for account_id...

gefragt vor 2 Jahren391 Aufrufe
2 Antworten
1

This can be done using the below query:

CREATE EXTERNAL TABLE IF NOT EXISTS tgwflowlogspqttbl ( version int, resource_type string, account_id string, tgw_id string, tgw_attachment_id string, tgw_src_vpc_account_id string, tgw_dst_vpc_account_id string, tgw_src_vpc_id string, tgw_dst_vpc_id string, tgw_src_subnet_id string, tgw_dst_subnet_id string, tgw_src_eni string, tgw_dst_eni string, tgw_src_az_id string, tgw_dst_az_id string, tgw_pair_attachment_id string, srcaddr string, dstaddr string, srcpor int, dstport int, protocol bigint, packet bigint, bytes bigint, start bigint, end bigint, log_status string, type string, packets_lost_no_route bigint, packets_lost_blackhole bigint, packets_lost_mtu_exceeded bigint, packets_lost_ttl_expired bigint, tcp_flags int, region string, flow_direction string, pkt_src_aws_service string, pkt_dst_aws_service string) ROW FORMAT SERDE 'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe' STORED AS INPUTFORMAT 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat' LOCATION 's3://bucket/tgw-logs-parquet/' TBLPROPERTIES ( 'skip.header.line.count'='1' )

AWS
beantwortet vor 5 Monaten
0

Hi There

Querying TGW flow logs with Athena is not currently supported

See https://docs.aws.amazon.com/vpc/latest/tgw/tgw-flow-logs.html

You can search TGW flow logs using Cloudwatch

See https://docs.aws.amazon.com/vpc/latest/tgw/working-with-flow-logs.html#search-flow-log-records

profile pictureAWS
EXPERTE
Matt-B
beantwortet vor 2 Jahren

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen