AWS Glue connect with on premise DB

Question

I am creating sample glue job in aws console using interactive session  to connect to on-premise oracle database but getting error. Same code when I am running from Docker is working fine.

tried with two versions of URL

jdbc:oracle:thin:@//hostname:1521/dbname

jdbc:oracle:thin://@hostname:1521/dbname

`url='mentioned above '
user='xxgmdmadm'
password='**********'
dbtable='xxgmdmadm.t10'
df_monitering=spark.read.format('jdbc')
.option('url',url)
.option('dbtable',dbtable)
.option('user',user)
.option('password',password).load()`

I am getting below error 
`Py4JJavaError: An error occurred while calling o80.load.
: java.sql.SQLRecoverableException: IO Error: Unknown host specified 
	at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:489)
	at oracle.jdbc.driver.PhysicalConnection.(PhysicalConnection.java:553)
	at oracle.jdbc.driver.T4CConnection.(T4CConnection.java:254)
	at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:32)
	at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:528)
	at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$createConnectionFactory$1.apply(JdbcUtils.scala:63)
	at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$createConnectionFactory$1.apply(JdbcUtils.scala:54)
	at org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD$.resolveTable(JDBCRDD.scala:56)
	at org.apache.spark.sql.execution.datasources.jdbc.JDBCRelation$.getSchema(JDBCRelation.scala:210)
	at org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider.createRelation(JdbcRelationProvider.scala:35)
	at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:318)
	at org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:223)
	at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:211)
	at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:167)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
	at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
	at py4j.Gateway.invoke(Gateway.java:282)
	at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
	at py4j.commands.CallCommand.execute(CallCommand.java:79)
	at py4j.GatewayConnection.run(GatewayConnection.java:238)
	at java.lang.Thread.run(Thread.java:748)
Caused by: oracle.net.ns.NetException: Unknown host specified 
	at oracle.net.resolver.HostnameNamingAdapter.resolve(HostnameNamingAdapter.java:191)
	at oracle.net.resolver.NameResolver.resolveName(NameResolver.java:133)
	at oracle.net.resolver.AddrResolution.resolveAndExecute(AddrResolution.java:431)
	at oracle.net.ns.NSProtocol.establishConnection(NSProtocol.java:693)
	at oracle.net.ns.NSProtocol.connect(NSProtocol.java:251)
	at oracle.jdbc.driver.T4CConnection.connect(T4CConnection.java:1140)
	at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:340)
	... 24 more`

Answer

Hello,

Please check the troubleshooting links here. I found the error that you mentioned in the AWS Reference Links.

Link- https://docs.aws.amazon.com/glue/latest/dg/glue-troubleshooting-errors.html

You may be trying to parametize AWS Glue jobs to apply the same transformation/logic on different datasets in Amazon S3. You want to track processed files on the locations provided. When you run the same job on the same source bucket and write to the same/different destination concurrently (concurrency >1) the job fails with this error:

Solution: set concurrency to 1 or don't run the job concurrently.

Currently AWS Glue bookmarks don't support concurrent job runs and commits will fail.

AWS Glue connect with on premise DB

Conteúdo relevante