Questions tagged with Database
Content language: English
Sort by most recent
RDS snapshot cleanup request - stuck in creating at 0% for 8+ months
Hi there -- AWS bug appears to have resulted in an inoperable RDS snapshot stuck at 0% for many months. Can you kindly work some voodoo magic to destroy database-1-final-snapshot in us-east-1 as neither API or management console allows us to achieve this goal on our end. Appreciate the time.
How do I return a COUNT of items with AppSync?
Hi all, quick question here, let's say I have 2 tables in DynamoDB: CITIES and COUNTRIES. I use AppSync to make queries to these tables and I want to return the COUNT of items (i.e. cities) in "Spain", how can I do that? Also, how can I return the total count of items in a table? Thanks in advance!
PostgreSQL Connection to RDS from external server - Connection errors but works from other sources
I have a Lambda Python function connecting via psycopg2 to a PostgreSQL db instance running RDS. The Lambda connects absolutely fine (Lambda and RDS both in EU-West-2 region) I can also connect to the PostgreSQL via PgAdmin4 from a local development system and other developers can also access from other locations/IPs via PGAdmin with no problem. I can also connect a simple psycopg2 connect and query script from my local desktop here. Therefore I know RDS is accepting and responding to externally-sourced psycopg2 connections and queries. HOWEVER, when I upload the same simple connect script to my web server (OVH - based in France if of any relevance), running equivalent Python and psycopg2 etc., the connection fails with the standard psycopg2 error response from the Python: `Error raised: connection to server at "xxxxxxxx.yyyyyyyyyy.eu-west-2.rds.amazonaws.com" (ppp.qqq.rrr.ssss), port 5432 failed: Connection refused Is the server running on that host and accepting TCP/IP connections?` I've tweaked the Security Group settings to permit anything from anywhere etc and still no joy. PostgreSQL in the RDS seems to have listening on * which seems necessary to permit connections under certain circumstances. What is the subtlety in the differing sources that means such a connection from the OVH web server won't work; I can't find anything in the docs that seems to link to this issue and there's nothing obvious mis-configured on the server-side.. Any responses gratefully received.
RDS "Failed to start" when i changed the parameter group
with the default parameter group, there is no problem to pause and start RDS but with the custom parameter group, i face "Failed to start" error everytime i pause and start RDS after the "Failed to start" error, RDS backup and reboot the database automatically, and it works fine at the end. - i've changed the parameter group from "default.mysql8.0" to the custom parameter group - i haven't changed any parameters of the new parameter group, every parameters is same with the default parameter group this is the log of the lastest pause and start, i have no idea what is the problem, thanks for the any advice `2022-12-03T16:31:42.090150Z 0 [Warning] [MY-011068] [Server] The syntax 'log_s lave_updates' is deprecated and will be removed in a future release. Please use log_replica_updates instead. 2022-12-03T16:31:42.090159Z 0 [Warning] [MY-011069] [Server] The syntax '--m aster-info-repository' is deprecated and will be removed in a future release. 2022-12-03T16:31:42.090164Z 0 [Warning] [MY-011069] [Server] The syntax '--m aster-info-repository' is deprecated and will be removed in a future release. 2022-12-03T16:31:42.090220Z 0 [Warning] [MY-011069] [Server] The syntax '--relay-log-info-file' is deprecated and will be removed in a future release. 2022-12-03T16:31:42.090225Z 0 [Warning] [MY-011069] [Server] The syntax '--relay-log-info-repository' is deprecated and will be removed in a future release. 2022-12-03T16:31:42.090252Z 0 [Warning] [MY-011068] [Server] The syntax 'skip_s lave_start' is deprecated and will be removed in a future release. Please use skip_replica_start instead. 2022-12-03T16:31:42.090259Z 0 [Warning] [MY-011068] [Server] The syntax 's lave_exec_mode' is deprecated and will be removed in a future release. Please use replica_exec_mode instead. 2022-12-03T16:31:42.090265Z 0 [Warning] [MY-011068] [Server] The syntax 's lave_load_tmpdir' is deprecated and will be removed in a future release. Please use replica_load_tmpdir instead. 2022-12-03T16:31:42.090437Z 0 [Warning] [MY-010101] [Server] Insecure configuration for --secure-file-priv: Location is accessible to all OS users. Consider choosing a different directory. 2022-12-03T16:31:42.090470Z 0 [Warning] [MY-010918] [Server] 'default_authentication_plugin' is deprecated and will be removed in a future release. Please use authentication_policy instead. 2022-12-03T16:31:42.090478Z 0 [System] [MY-010116] [Server] /rdsdbbin/mysql/bin/mysqld (mysqld 8.0.30) starting as process 731 2022-12-03T16:31:42.099878Z 0 [Warning] [MY-013907] [InnoDB] Deprecated configuration parameters innodb_log_file_size and/or innodb_log_files_in_group have been used to compute innodb_redo_log_capacity=268435456. Please use innodb_redo_log_capacity instead. 2022-12-03T16:31:42.101840Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started. 2022-12-03T16:31:42.104547Z 1 [Warning] [MY-012191] [InnoDB] Scan path '/rdsdbdata/db/innodb' is ignored because it is a sub-directory of '/rdsdbdata/db/ 2022-12-03T16:31:42.513654Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended. 2022-12-03T16:31:42.827606Z 0 [Warning] [MY-013414] [Server] Server SSL certificate doesn't verify: unable to get issuer certificate 2022-12-03T16:31:42.827781Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel. 2022-12-03T16:31:42.887455Z 0 [System] [MY-010931] [Server] /rdsdbbin/mysql/bin/mysqld: ready for connections. Version: '8.0.30' socket: '/tmp/mysql.sock' port: 3306 Source distribution. 2022-12-03T16:34:08.192155Z 11 [Warning] [MY-010055] [Server] IP address '22.214.171.124' could not be resolved: Name or service not known 2022-12-03T16:34:08.192271Z 12 [Warning] [MY-010055] [Server] IP address '126.96.36.199' could not be resolved: Name or service not known 2022-12-03T16:34:08.192151Z 10 [Warning] [MY-010055] [Server] IP address '188.8.131.52' could not be resolved: Name or service not known 2022-12-03T16:34:08.192487Z 9 [Warning] [MY-010055] [Server] IP address '184.108.40.206' could not be resolved: Name or service not known `
How do you change the MASTER_HEARTBEAT_PERIOD on an Aurora MySQL version 2 (with MySQL 5.7 compatibility) RDS database?
AWS is forcing us to upgrade our RDS databases from MySQL 5.6 to 5.7 before February 28, 2023. MySQL 5.7 automatically decreases the slave_net_timeout from 3600 sec to 60 sec which we agree with. However, the MASTER_HEARTBEAT_PERIOD remained at 1800 sec, which causes replication problems because it should always be half of the slave_net_timeout ([MySQL Documentation](https://dev.mysql.com/doc/mysql-installation-excerpt/5.7/en/upgrading-from-previous-series.html)). > ![Enter image description here](/media/postImages/original/IM28oWh61pRsyE_2bTwP7Cqg) We tried the following command (*CHANGE MASTER TO MASTER_HEARTBEAT_PERIOD = 30;*) but it returns an error (*Error Code: 1227. Access denied; you need (at least one of) the SUPER privilege(s) for this operation*). The problem is that AWS explicitly doesn't give users SUPER privileges ([AWS Documentation](https://aws.amazon.com/premiumsupport/knowledge-center/rds-mysql-functions/)). > ![Enter image description here](/media/postImages/original/IMgxcRkXAkRG2mzQb20edGRQ) How do we change the MASTER_HEARTBEAT_PERIOD if it requires SUPER privileges and AWS doesn’t give it to us?
Athena query to count rows for each table in Glue catalog
Hi, Following from this article: [Get record count for all tables in mysql database](https://www.tutorialspoint.com/get-record-count-for-all-tables-in-mysql-database#:~:text=To%20get%20the%20count%20of,The%20syntax%20is%20as%20follows.&text=mysql%3E%20SELECT%20SUM(TABLE_ROWS)%20%2D%3EFROM%20INFORMATION_SCHEMA.), is there an Athena on Presto version of the following MySQL query? ``` SELECT table_name , table_rows FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA = schema_name; ```
Weird DMS Error: "Replication Task Settings document error: Array element's unique identifier was not found, please provide Id for each and every element"
Trying to set up DMS between RDS MySQL and Redshift. If I try to edit the the Table Mappings from the GUI, I always get a weird error (that has zero Google results, except 1 spam page) `Replication Task Settings document error: Array element's unique identifier was not found, please provide Id for each and every element`. No idea what that means. Every rule has a unique name and ID. Not the biggest issue in the world, as it's usually okay if I make a new one, but it does make it hard to iterate. Any idea what the issue here could be? Thanks!
RDS for database size more than 16TB
As per AWS documentation the max size supported by AWS RDS DB is 16TB. My on-prem MS SQL DB is approx 25TB. So I cant use RDS. -what are different options available as I migrate this DB to AWS? - If I install MS SQL on EC2 instance, are there any Pro's and Con's - Is there any AWS recommended document for installing MS SQL on a EC2 RHEL instance? - For MS SQL in EC2, how to best design the backup and restore process?
"SCHEMA_NOT_FOUND" Error in Athena
I'm getting a "SCHEMA_NOT_FOUND" error when trying to preview a table in Athena. Using the same connection, data source and database, I'm able to preview the table in a Glue DataBrew project so I know the table is populated and accessible . The Athena query is using Athena 3.0 in us-west-1. Any help would be appreciated! ``` SCHEMA_NOT_FOUND: line 1:15: Schema 'exp' does not exist This query ran against the "EXP" database, unless qualified by the query. Please post the error message on our forum or contact customer support with Query Id: 5f17ea48-4f09-4285-b8d2-aa0be96abe38 ``` ![Enter image description here](/media/postImages/original/IMRvQW1esQQuGWrQ-7sKFhQw)
Hi all, I run a simple query on Athena using Athena engine version 3. ``` create table if not exists "test"."total_customer" as select * from "A_customer" union all select * from "B_customer" ``` And I faced this error: `HIVE_PATH_ALREADY_EXISTS: Target directory for table 'test.total_customer' already exists: s3://xxxx/total_customer/2022/11/27/tables/ef02d72d-75e2-4b14-b53f-95d591094cfa. You may need to manually clean the data at location 's3://xxxx/total_customer/2022/11/27/tables/ef02d72d-75e2-4b14-b53f-95d591094cfa' before retrying. Athena will not delete data in your account.` The weird thing is it works perfectly when I use UNION instead of UNION ALL. So I'm not sure if this is Athena engine version 3 error or the query error. Thank you guys for taking a look at my issue.