Athena, table is created by always shows 0 results and 0kb scanned

0

Hi,

New to Athena and I would like to query Classic ELB for IP addresses in an effort to identify some web scrapers (sort of the first step in identifying if we have a problem with scrapers or not). Anyway Athena seems to be a really easy to ingest our logs that we have saved to S3 without ever really thinking about it for three years.

I am following the guide available here:https://docs.aws.amazon.com/athena/latest/ug/elasticloadbalancer-classic-logs.html

I tried to create the table using the following SQL, it creates as expected but always shows 0 rows and 0kb scanned. I've subsquently tried more focussed folder paths (for example 1 days worth of logs instead of all the three years with the same output. I've tried the classic and application load balancer regex but I am sure it's the classic I need.

CREATE EXTERNAL TABLE IF NOT EXISTS elb_logs (
timestamp string,
elb_name string,
request_ip string,
request_port int,
backend_ip string,
backend_port int,
request_processing_time double,
backend_processing_time double,
client_response_time double,
elb_response_code string,
backend_response_code string,
received_bytes bigint,
sent_bytes bigint,
request_verb string,
url string,
protocol string,
user_agent string,
ssl_cipher string,
ssl_protocol string
)
ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.RegexSerDe'
WITH SERDEPROPERTIES (
'serialization.format' = '1',
'input.regex' = '([^ ]) ([^ ]) ([^ ]):([0-9]) ([^ ])[:-]([0-9]) ([-.0-9]) ([-.0-9]) ([-.0-9]) (|[-0-9]) (-|[-0-9]) ([-0-9]) ([-0-9]) \"([^ ]) ([^ ]) (- |[^ ])\" ("[^"]") ([A-Z0-9-]+) ([A-Za-z0-9.-])$' )
LOCATION 's3://MYBUCKETNAME/AWSLogs/AWSACCOUNTID/elasticloadbalancing/'

There are obviously files in the s3 bucket, I am using an IAM user with Full AWS Permissions and

Any thoughts?

Everything seems to suggest the path is not right - but I think...it i...s Thanks for your patience.

Thanks

質問済み 5年前403ビュー
2回答
0

Well it's started working - presumably it just takes a little time and I was being impatient. Sorry to bother everyone!

回答済み 5年前
0

I made no changes, 25 hours previewed the table and the data appeared.

回答済み 5年前

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

質問に答えるためのガイドライン

関連するコンテンツ