By using AWS re:Post, you agree to the Terms of Use
Questions in Migration & Transfer
Sort by most recent

Browse through the questions and answers listed below or filter and sort to narrow down your results.

Sending AS2 messages between two AWS Transfer Family servers

I've set up AWS Transfer Family servers in two different regions to test the sending functionality. However, even though the VPC is created, sending messages fail with either UNABLE_TO_CONNECT_TO_REMOTE_HOST_OR_IP or "File path not found". I'm using S3 for the document to send. I've checked the IP address with a different program (Mendelson AS2) and it's able to connect fine. It even was able to send a test document. Despite that, when sending through a lambda function, it fails. A few things tried: * Checking permissions: I'm able to connect and describe the server, the connectors, etc with no problem so it's not that * Connector with the wrong URL: I used the same URL as the URL in Mendelson with the port attached at the end (http:/s-xxx:5080 in the format specified in [1] with the region). I also tried the URL without the port specified and that didn't work either * Region issue: I thought the mismatch between the region could be an issue since the lambda was set in us-west-1 while the as2 server I was sending to is in us-east2 so I created a different connector and had it send to itself in the same region. Still the same error with being unable to connect * Checked the cloudwatch logs: It actually reports that everything sent successfully with a 200 code Weird things noticed: * After the lambda is triggered, it creates the expected failed and processing folder but after the first few times, it no longer saves the results. I get a .cms file and a .json file sometimes but not every time, even though the cloudwatch logs are correctly created every time. * The failed and processed folders somehow got created a folder above rather than the folder the file was uploaded to. (e.g. the folder structure is bucket/folder 1/folder2/folder 3 and the uploaded file was in folder3. However, the failed and processing folders were created in folder2 instead of the expected folder3. This happened just once though. Additional question: I can upload this as a different question if needed but since it's related to my issue, I figured I'd put it here as well * What's the transfer id for? Is that supposed to be the execution id? There doesn't seem to be an option to view the results of the transfer in the documentation [2]. References: [1] https://docs.aws.amazon.com/transfer/latest/userguide/as2-end-to-end-example.html#as2-create-connector-example [2] https://docs.aws.amazon.com/transfer/latest/userguide/API_StartFileTransfer.html
0
answers
0
votes
3
views
asked 4 hours ago

DMS Postgres Source Error: No tables were found at task initialization

I am trying to setup AWS DMS using an RDS Postgres instance as a source. However, I keep getting the following error when starting the replication task: > Last Error No tables were found at task initialization. Either the selected table(s) or schemas(s) no longer exist or no match was found for the table selection pattern(s). If you would like to start a Task that does not initially capture any tables, set Task Setting FailOnNoTablesCaptured to false and restart task. Stop Reason FATAL_ERROR Error Level FATAL I followed the [DMS guide](https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.PostgreSQL.html#CHAP_Source.PostgreSQL.RDSPostgreSQL) and my source connection test is successful. I am using the master account and have verified the pglogical plugin is installed. For my Task selection rules, I have tried using the wildcard (`"schema-name": "%", "table-name": "%"`) and targeting specific schema and tables with no success. I have also tried all 3 migration types: full-load, cdc, and full-load-and-cdc. Here is my complete task configuration: ``` { "Logging": { "EnableLogging": false, "LogComponents": [ { "Severity": "LOGGER_SEVERITY_DEFAULT", "Id": "TRANSFORMATION" }, { "Severity": "LOGGER_SEVERITY_DEFAULT", "Id": "SOURCE_UNLOAD" }, { "Severity": "LOGGER_SEVERITY_DEFAULT", "Id": "IO" }, { "Severity": "LOGGER_SEVERITY_DEFAULT", "Id": "TARGET_LOAD" }, { "Severity": "LOGGER_SEVERITY_DEFAULT", "Id": "PERFORMANCE" }, { "Severity": "LOGGER_SEVERITY_DEFAULT", "Id": "SOURCE_CAPTURE" }, { "Severity": "LOGGER_SEVERITY_DEFAULT", "Id": "SORTER" }, { "Severity": "LOGGER_SEVERITY_DEFAULT", "Id": "REST_SERVER" }, { "Severity": "LOGGER_SEVERITY_DEFAULT", "Id": "VALIDATOR_EXT" }, { "Severity": "LOGGER_SEVERITY_DEFAULT", "Id": "TARGET_APPLY" }, { "Severity": "LOGGER_SEVERITY_DEFAULT", "Id": "TASK_MANAGER" }, { "Severity": "LOGGER_SEVERITY_DEFAULT", "Id": "TABLES_MANAGER" }, { "Severity": "LOGGER_SEVERITY_DEFAULT", "Id": "METADATA_MANAGER" }, { "Severity": "LOGGER_SEVERITY_DEFAULT", "Id": "FILE_FACTORY" }, { "Severity": "LOGGER_SEVERITY_DEFAULT", "Id": "COMMON" }, { "Severity": "LOGGER_SEVERITY_DEFAULT", "Id": "ADDONS" }, { "Severity": "LOGGER_SEVERITY_DEFAULT", "Id": "DATA_STRUCTURE" }, { "Severity": "LOGGER_SEVERITY_DEFAULT", "Id": "COMMUNICATION" }, { "Severity": "LOGGER_SEVERITY_DEFAULT", "Id": "FILE_TRANSFER" } ], "CloudWatchLogGroup": null, "CloudWatchLogStream": null }, "StreamBufferSettings": { "StreamBufferCount": 3, "CtrlStreamBufferSizeInMB": 5, "StreamBufferSizeInMB": 8 }, "ErrorBehavior": { "FailOnNoTablesCaptured": true, "ApplyErrorUpdatePolicy": "LOG_ERROR", "FailOnTransactionConsistencyBreached": false, "RecoverableErrorThrottlingMax": 1800, "DataErrorEscalationPolicy": "SUSPEND_TABLE", "ApplyErrorEscalationCount": 0, "RecoverableErrorStopRetryAfterThrottlingMax": true, "RecoverableErrorThrottling": true, "ApplyErrorFailOnTruncationDdl": false, "DataTruncationErrorPolicy": "LOG_ERROR", "ApplyErrorInsertPolicy": "LOG_ERROR", "EventErrorPolicy": "IGNORE", "ApplyErrorEscalationPolicy": "LOG_ERROR", "RecoverableErrorCount": -1, "DataErrorEscalationCount": 0, "TableErrorEscalationPolicy": "STOP_TASK", "RecoverableErrorInterval": 5, "ApplyErrorDeletePolicy": "IGNORE_RECORD", "TableErrorEscalationCount": 0, "FullLoadIgnoreConflicts": true, "DataErrorPolicy": "LOG_ERROR", "TableErrorPolicy": "SUSPEND_TABLE" }, "TTSettings": { "TTS3Settings": null, "TTRecordSettings": null, "EnableTT": false }, "FullLoadSettings": { "CommitRate": 10000, "StopTaskCachedChangesApplied": false, "StopTaskCachedChangesNotApplied": false, "MaxFullLoadSubTasks": 8, "TransactionConsistencyTimeout": 600, "CreatePkAfterFullLoad": false, "TargetTablePrepMode": "DROP_AND_CREATE" }, "TargetMetadata": { "ParallelApplyBufferSize": 0, "ParallelApplyQueuesPerThread": 0, "ParallelApplyThreads": 0, "TargetSchema": "", "InlineLobMaxSize": 0, "ParallelLoadQueuesPerThread": 0, "SupportLobs": false, "LobChunkSize": 0, "TaskRecoveryTableEnabled": false, "ParallelLoadThreads": 0, "LobMaxSize": 0, "BatchApplyEnabled": false, "FullLobMode": false, "LimitedSizeLobMode": false, "LoadMaxFileSize": 0, "ParallelLoadBufferSize": 0 }, "BeforeImageSettings": null, "ControlTablesSettings": { "historyTimeslotInMinutes": 5, "HistoryTimeslotInMinutes": 5, "StatusTableEnabled": false, "SuspendedTablesTableEnabled": false, "HistoryTableEnabled": false, "ControlSchema": "", "FullLoadExceptionTableEnabled": false }, "LoopbackPreventionSettings": null, "CharacterSetSettings": null, "FailTaskWhenCleanTaskResourceFailed": false, "ChangeProcessingTuning": { "StatementCacheSize": 50, "CommitTimeout": 1, "BatchApplyPreserveTransaction": true, "BatchApplyTimeoutMin": 1, "BatchSplitSize": 0, "BatchApplyTimeoutMax": 30, "MinTransactionSize": 1000, "MemoryKeepTime": 60, "BatchApplyMemoryLimit": 500, "MemoryLimitTotal": 1024 }, "ChangeProcessingDdlHandlingPolicy": { "HandleSourceTableDropped": true, "HandleSourceTableTruncated": true, "HandleSourceTableAltered": true }, "PostProcessingRules": null } ```
1
answers
0
votes
9
views
asked 2 days ago