EMR Presto Use External Hive RDS Metastore

0

Hi, I have an EMR Presto Cluster on EC2 and an external hive metastore on a RDS Mysql instance in AWS. I've read some documentation about configuring an external metastore on Amazon Glue but nothing about how to enable the external RDS metastore for Presto. I was able to access the MySQL RDS database from the EMR Presto cluster, so it is not a network issue, and also tried several configurations, example:

[
    {
      "Classification": "hive-site",
      "Properties": {
        "hive.metastore.client.factory.class": "org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClientFactory",
        "javax.jdo.option.ConnectionURL": "jdbc:mysql://[instance]:[port]/[database]?createDatabaseIfNotExist=true",
        "javax.jdo.option.ConnectionDriverName": "org.mariadb.jdbc.Driver",
        "javax.jdo.option.ConnectionUserName": "[user]",
        "javax.jdo.option.ConnectionPassword": "[password]",
        "hive.metastore.uris": "thrift://[instance]:[port]/[database]"
      }
    }
]
[
  {
    "Classification": "presto-connector-hive",
    "Properties": {
      "connection-url": "jdbc:mysql://[instance]:[port]/[database]",
      "connection-user": "[user]",
      "connection-password": "[password]",
    }
  }
]

None of them have worked so far. What would be the required Presto configuration? Thanks in advance.

asked 2 years ago430 views
1 Answer
0

looks like you were trying incorrect classification for the properties you want to set. "Classification": "presto-connector-hive"

you might need to use presto-connector-mysql Classification instead to set properties related to Presto mysql Connector https://prestodb.io/docs/current/connector/mysql.html https://docs.aws.amazon.com/emr/latest/ReleaseGuide/presto-adding-db-connectors.html

presto-connector-hive is used to set following properties (like setting hive metastore etc) https://prestodb.io/docs/current/connector/hive.html if this metastore is configured to access External mysql Metastore DB, then presto would be able to access that as Alternate approach. https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-hive-metastore-external.html

possible classification list per release label : https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-5360-release.html#emr-5360-class

profile pictureAWS
answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions