EMR Presto Use External Hive RDS Metastore

0

Hi, I have an EMR Presto Cluster on EC2 and an external hive metastore on a RDS Mysql instance in AWS. I've read some documentation about configuring an external metastore on Amazon Glue but nothing about how to enable the external RDS metastore for Presto. I was able to access the MySQL RDS database from the EMR Presto cluster, so it is not a network issue, and also tried several configurations, example:

[
    {
      "Classification": "hive-site",
      "Properties": {
        "hive.metastore.client.factory.class": "org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClientFactory",
        "javax.jdo.option.ConnectionURL": "jdbc:mysql://[instance]:[port]/[database]?createDatabaseIfNotExist=true",
        "javax.jdo.option.ConnectionDriverName": "org.mariadb.jdbc.Driver",
        "javax.jdo.option.ConnectionUserName": "[user]",
        "javax.jdo.option.ConnectionPassword": "[password]",
        "hive.metastore.uris": "thrift://[instance]:[port]/[database]"
      }
    }
]
[
  {
    "Classification": "presto-connector-hive",
    "Properties": {
      "connection-url": "jdbc:mysql://[instance]:[port]/[database]",
      "connection-user": "[user]",
      "connection-password": "[password]",
    }
  }
]

None of them have worked so far. What would be the required Presto configuration? Thanks in advance.

gefragt vor 2 Jahren439 Aufrufe
1 Antwort
0

looks like you were trying incorrect classification for the properties you want to set. "Classification": "presto-connector-hive"

you might need to use presto-connector-mysql Classification instead to set properties related to Presto mysql Connector https://prestodb.io/docs/current/connector/mysql.html https://docs.aws.amazon.com/emr/latest/ReleaseGuide/presto-adding-db-connectors.html

presto-connector-hive is used to set following properties (like setting hive metastore etc) https://prestodb.io/docs/current/connector/hive.html if this metastore is configured to access External mysql Metastore DB, then presto would be able to access that as Alternate approach. https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-hive-metastore-external.html

possible classification list per release label : https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-5360-release.html#emr-5360-class

profile pictureAWS
beantwortet vor einem Jahr

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen