By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to dynamodb.ap-south-1.amazonaws.com:443

0

I have a running MSK connector which talks with a Kafka running on ec2-instance in a created VPN with public subnets. when the dynamo connector is trying to access the dynamodb table getting the below error:

[Worker-02698e88eb6557581] com.amazonaws.SdkClientException: Unable to execute HTTP request: Connect to dynamodb.ap-south-1.amazonaws.com:443 [dynamodb.ap-south-1.amazonaws.com/52.94.20.64] failed: connect timed out
[Worker-02698e88eb6557581] 	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleRetryableException(AmazonHttpClient.java:1207)
[Worker-02698e88eb6557581] 	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1153)
[Worker-02698e88eb6557581] 	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:802)
[Worker-02698e88eb6557581] 	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:770)
[Worker-02698e88eb6557581] 	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:744)
[Worker-02698e88eb6557581] 	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:704)
[Worker-02698e88eb6557581] 	at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:686)
[Worker-02698e88eb6557581] 	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:550)
[Worker-02698e88eb6557581] 	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:530)
[Worker-02698e88eb6557581] 	at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.doInvoke(AmazonDynamoDBClient.java:6214)
[Worker-02698e88eb6557581] 	at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.invoke(AmazonDynamoDBClient.java:6181)
[Worker-02698e88eb6557581] 	at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.executeDescribeTable(AmazonDynamoDBClient.java:2247)
[Worker-02698e88eb6557581] 	at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.describeTable(AmazonDynamoDBClient.java:2211)
[Worker-02698e88eb6557581] 	at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.describeTable(AmazonDynamoDBClient.java:2259)
[Worker-02698e88eb6557581] 	at com.trustpilot.connector.dynamodb.aws.ConfigTablesProvider.getConsumableTables(ConfigTablesProvider.java:28)
[Worker-02698e88eb6557581] 	at com.trustpilot.connector.dynamodb.DynamoDBSourceConnector.taskConfigs(DynamoDBSourceConnector.java:120)
[Worker-02698e88eb6557581] 	at org.apache.kafka.connect.runtime.Worker.connectorTaskConfigs(Worker.java:359)
[Worker-02698e88eb6557581] 	at org.apache.kafka.connect.runtime.distributed.DistributedHerder.reconfigureConnector(DistributedHerder.java:1469)
[Worker-02698e88eb6557581] 	at org.apache.kafka.connect.runtime.distributed.DistributedHerder.reconfigureConnectorTasksWithRetry(DistributedHerder.java:1407)
[Worker-02698e88eb6557581] 	at org.apache.kafka.connect.runtime.distributed.DistributedHerder.access$1000(DistributedHerder.java:130)
[Worker-02698e88eb6557581] 	at org.apache.kafka.connect.runtime.distributed.DistributedHerder$12.call(DistributedHerder.java:1359)
[Worker-02698e88eb6557581] 	at org.apache.kafka.connect.runtime.distributed.DistributedHerder$12.call(DistributedHerder.java:1353)
[Worker-02698e88eb6557581] 	at org.apache.kafka.connect.runtime.distributed.DistributedHerder.tick(DistributedHerder.java:399)
[Worker-02698e88eb6557581] 	at org.apache.kafka.connect.runtime.distributed.DistributedHerder.run(DistributedHerder.java:317)
[Worker-02698e88eb6557581] 	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
[Worker-02698e88eb6557581] 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[Worker-02698e88eb6557581] 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[Worker-02698e88eb6557581] 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[Worker-02698e88eb6557581] 	at java.base/java.lang.Thread.run(Thread.java:829)
[Worker-02698e88eb6557581] Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to dynamodb.ap-south-1.amazonaws.com:443 [dynamodb.ap-south-1.amazonaws.com/52.94.20.64] failed: connect timed out
[Worker-02698e88eb6557581] 	at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151)
[Worker-02698e88eb6557581] 	at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376)
[Worker-02698e88eb6557581] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[Worker-02698e88eb6557581] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[Worker-02698e88eb6557581] 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[Worker-02698e88eb6557581] 	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
[Worker-02698e88eb6557581] 	at com.amazonaws.http.conn.ClientConnectionManagerFactory$Handler.invoke(ClientConnectionManagerFactory.java:76)
[Worker-02698e88eb6557581] 	at com.amazonaws.http.conn.$Proxy42.connect(Unknown Source)
[Worker-02698e88eb6557581] 	at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393)
[Worker-02698e88eb6557581] 	at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
[Worker-02698e88eb6557581] 	at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
[Worker-02698e88eb6557581] 	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
[Worker-02698e88eb6557581] 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
[Worker-02698e88eb6557581] 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
[Worker-02698e88eb6557581] 	at com.amazonaws.http.apache.client.impl.SdkHttpClient.execute(SdkHttpClient.java:72)
[Worker-02698e88eb6557581] 	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1331)
[Worker-02698e88eb6557581] 	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1145)
[Worker-02698e88eb6557581] 	... 27 more
[Worker-02698e88eb6557581] Caused by: java.net.SocketTimeoutException: connect timed out
[Worker-02698e88eb6557581] 	at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
[Worker-02698e88eb6557581] 	at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:412)
[Worker-02698e88eb6557581] 	at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:255)
[Worker-02698e88eb6557581] 	at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:237)
[Worker-02698e88eb6557581] 	at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
[Worker-02698e88eb6557581] 	at java.base/java.net.Socket.connect(Socket.java:609)
[Worker-02698e88eb6557581] 	at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:368)
[Worker-02698e88eb6557581] 	at com.amazonaws.http.conn.ssl.SdkTLSSocketFactory.connectSocket(SdkTLSSocketFactory.java:142)
[Worker-02698e88eb6557581] 	at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142)
[Worker-02698e88eb6557581] 	... 43 more

I believe the MSK connector be running in the same VPC, I tried pinging google in the ec2 hosted in the subnet, pinging works.

  • Leeroy Hannigan EXPERT
    7 months ago

    ping is not a valid test as it uses ICMP. Make sure you can connect to the internet on port 443.

Topics
Networking & Content DeliveryDatabaseAnalytics
Tags
Amazon VPCAmazon DynamoDBAmazon Managed Streaming for Apache Kafka (Amazon MSK)
Language
English
profile picture
Ritik
asked 7 months ago314 views
2 Answers
0

Yes both the subnets are public, I initially followed the docs and tried with a public subnet and a private subnet along with NAT. It showed same issue.

profile picture
Ritik
answered 7 months ago
-1

ok, I wrote a small java code making api call,

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;

public class TodoClient {
    public static void main(String[] args) {
        sampleCall();
    }

    public static void sampleCall() {
        try {
            URL url = new URL("https://jsonplaceholder.typicode.com/todos/1");

            HttpURLConnection connection = (HttpURLConnection) url.openConnection();

            connection.setRequestMethod("GET");

            int responseCode = connection.getResponseCode();
            System.out.println("Response Code: " + responseCode);

            if (responseCode == HttpURLConnection.HTTP_OK) {
                BufferedReader reader = new BufferedReader(new InputStreamReader(connection.getInputStream()));
                String line;
                StringBuilder response = new StringBuilder();

                while ((line = reader.readLine()) != null) {
                    response.append(line);
                }
                reader.close();

                System.out.println("Response Data:");
                System.out.println(response.toString());
            } else {
                System.out.println("Failed to fetch data. Response code: " + responseCode);
            }

            // Close the connection
            connection.disconnect();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

executed on all the machines, the code was giving correct ouput, when similar code was running in an msk connector, it threw me Connection Timeout error.

profile picture
Ritik
answered 7 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content