Database

Hi, I am a little confused on what's required for this setup to work. I have two account Tooling account(Account A) and a Dev account(Account B). Account B has a db cluster(private isolated subnet) and have configured its security group to allow access from IP address associated with target region of CODEBUILD - https://ip-ranges.amazonaws.com/ip-ranges.json This of course is not sufficient alone since db cluster is in a vpc and not just that- different account as well. ``` Error: connect EHOSTUNREACH <addressIp>3306 at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1187:16) { errno: -113, code: 'EHOSTUNREACH', syscall: 'connect', address: <addressIp> port: 3306, fatal: true } ``` Could you please point me to the steps I'd have to take in order to be able to connect to it from CodeBuild ? I cannot seem to have any relevant documentation on what I am trying to achieve, beside just VPC support section on AWS - https://docs.aws.amazon.com/codebuild/latest/userguide/vpc-support.html . I do have a bastion host so the team can connect to it locally. Do I have to setup another VPC in Tooling account and do VPC peering between VPC in Account A and Account B ? I am using CDK to deploy infrastructure and for the pipelines as well. Any suggestions are greatly appreciated.lg...

Hello, I am new on this subject, I would like to keep my database housed in the always operational cloud since in my area where I live the Internet connection is not always reliable, and I would like to know if it is possible to manter the base of the base of the base Data locally when I suffer an Internet cut, since my base is housed in the cloud from now thank you very much I am rookie clariflg...

Hello, I have issues with table export from PhpMyAdmin on the Amazon Lightsale instance. I have access to my website database PhpMyAdmin through Local Host and until yesterday I was normally downloading databases and uploading new ones, but since yesterday I can't download them in any of the browsers. I can normally execute all commands and everything working fine except data export any advice is welcome. I have few other servers and database or table export is normally working.lg...

We are using Global tables in multiple regions with a single region for write operations. For automatic failover to a different region, can we use something on the application layer to change the writer region? Something like DNS to pick a healhy region, can result in writes to multiple regions while DNS propagates which is not safe as we use Transactions (not supported across multiple regions).lg...

I have a set of files in my S3 bucket which have a delimiter ASCII 31 (unit separator). I am using a crawler to read these files and create the tables in AWS Glue catalog. I tried using the custom delimiter in the classifiers but with no luck since this is a non-printable character. What is the best way to incorporate this delimiter within a crawler?lg...

One of our DB instances on MySQL 8.0.29 was crashing with regular "DB instance restarted" in the logs. I tried modifying it to 8.0.30, but now it's stuck upgrading, with regular "Error restarting mysql: MySQL process is not running and the restart attempt failed." in the logs. The error log contains the following over and over again: ``` 2022-09-23T16:03:46.082336Z 0 [Warning] [MY-011068] [Server] The syntax 'log_slave_updates' is deprecated and will be removed in a future release. Please use log_replica_updates instead. 2022-09-23T16:03:46.082352Z 0 [Warning] [MY-011069] [Server] The syntax '--master-info-repository' is deprecated and will be removed in a future release. 2022-09-23T16:03:46.082362Z 0 [Warning] [MY-011069] [Server] The syntax '--master-info-repository' is deprecated and will be removed in a future release. 2022-09-23T16:03:46.082415Z 0 [Warning] [MY-011069] [Server] The syntax '--relay-log-info-file' is deprecated and will be removed in a future release. 2022-09-23T16:03:46.082425Z 0 [Warning] [MY-011069] [Server] The syntax '--relay-log-info-repository' is deprecated and will be removed in a future release. 2022-09-23T16:03:46.082457Z 0 [Warning] [MY-011068] [Server] The syntax 'skip_slave_start' is deprecated and will be removed in a future release. Please use skip_replica_start instead. 2022-09-23T16:03:46.082469Z 0 [Warning] [MY-011068] [Server] The syntax 'slave_exec_mode' is deprecated and will be removed in a future release. Please use replica_exec_mode instead. 2022-09-23T16:03:46.082483Z 0 [Warning] [MY-011068] [Server] The syntax 'slave_load_tmpdir' is deprecated and will be removed in a future release. Please use replica_load_tmpdir instead. 2022-09-23T16:03:46.082676Z 0 [Warning] [MY-010101] [Server] Insecure configuration for --secure-file-priv: Location is accessible to all OS users. Consider choosing a different directory. 2022-09-23T16:03:46.082703Z 0 [Warning] [MY-010918] [Server] 'default_authentication_plugin' is deprecated and will be removed in a future release. Please use authentication_policy instead. 2022-09-23T16:03:46.082719Z 0 [System] [MY-010116] [Server] /rdsdbbin/mysql/bin/mysqld (mysqld 8.0.29) starting as process 726 2022-09-23T16:03:46.085105Z 0 [Warning] [MY-010161] [Server] You need to use --log-bin to make --log-replica-updates work. 2022-09-23T16:03:46.089322Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started. 2022-09-23T16:03:46.095259Z 1 [Warning] [MY-012191] [InnoDB] Scan path '/rdsdbdata/db/innodb' is ignored because it is a sub-directory of '/rdsdbdata/db/ 2022-09-23T16:03:48.212117Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended. InnoDB: Progress in percents: 116:03:49 UTC - mysqld got signal 11 ; Most likely, you have hit a bug, but this error can also be caused by malfunctioning hardware. Thread pointer: 0x1518a5406000 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 15189fcc63e8 thread_stack 0x40000 /rdsdbbin/mysql/bin/mysqld(my_print_stacktrace(unsigned char const*, unsigned long)+0x2e) [0x2097dae] /rdsdbbin/mysql/bin/mysqld(print_fatal_signal(int)+0x2bb) [0x102057b] /rdsdbbin/mysql/bin/mysqld(handle_fatal_signal+0xa5) [0x1020625] /lib64/libpthread.so.0(+0x118e0) [0x1518c0d458e0] /rdsdbbin/mysql/bin/mysqld(page_cur_search_with_match_bytes(buf_block_t const*, dict_index_t const*, dtuple_t const*, page_cur_mode_t, unsigned long*, unsigned long*, unsigned long*, unsigned long*, page_cur_t*)+0x190) [0x22d69b0] /rdsdbbin/mysql/bin/mysqld(btr_cur_search_to_nth_level(dict_index_t*, unsigned long, dtuple_t const*, page_cur_mode_t, unsigned long, btr_cur_t*, unsigned long, char const*, unsigned long, mtr_t*)+0x1ec2) [0x2406642] /rdsdbbin/mysql/bin/mysqld(row_search_on_row_ref(btr_pcur_t*, unsigned long, dict_table_t*, dtuple_t const*, mtr_t*)+0xd2) [0x233d2c2] /rdsdbbin/mysql/bin/mysqld(row_undo_search_clust_to_pcur(undo_node_t*)+0x15f) [0x234bd4f] /rdsdbbin/mysql/bin/mysqld(row_undo_mod(undo_node_t*, que_thr_t*)+0x18f) [0x258461f] /rdsdbbin/mysql/bin/mysqld(row_undo_step(que_thr_t*)+0x52) [0x234c542] /rdsdbbin/mysql/bin/mysqld(que_run_threads(que_thr_t*)+0x5c8) [0x22f3da8] /rdsdbbin/mysql/bin/mysqld(trx_rollback_or_clean_recovered(bool)+0x9a5) [0x23a4475] /rdsdbbin/mysql/bin/mysqld(trx_recovery_rollback_thread()+0x1c) [0x23a53fc] /rdsdbbin/mysql/bin/mysqld(std::thread::_State_impl<std::thread::_Invoker<std::tuple<Detached_thread, void (*)()> > >::_M_run()+0xb0) [0x2296fc0] /rdsdbbin/mysql/bin/mysqld() [0x2baabf0] /lib64/libpthread.so.0(+0x744b) [0x1518c0d3b44b] /lib64/libc.so.6(clone+0x3f) [0x1518c052040f] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (0): Connection ID (thread ID): 0 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. ``` Is there any way to fix this, or are we going to have to delete / recreate the instance?lg...

One of our RDS instance went into recovery by itself without any notification/admin interactions There were 1. nothing meaningful in the event logs, only indicate it went into recovery and restarted 2. nothing logged in the MySQL error logs either https://docs.google.com/spreadsheets/d/1H4BG-qbmvbfEGV5H9aL28cjuWIeVqMnrryDEuUlynzM/edit#gid=0 How do we find out what happened? (ex: if underlying hardware indeed failed?)lg...

I would like to know if there is a way to know if an iam user logs in to quicksight; to know the time and which panel he/she logs in to.lg...

I have a non-public DB Cluster & a Proxy (Aurora RDS PostgreSQL Serverless v2 13.7). It's in a VPC with private subnets. The Security Group is same for both Cluster and Proxy. Inbound rule allows TCP for Self Referenced Security Group on 5432 port. Outbound rule allows all traffic (0.0.0.0/0). I have a Lambda function (Python) in the same VPC, subnets and Security Group. I have also created an IAM user and attached a Policy for rds-db:connect. Likewise, I have a similar policy attached to the Lambda also for rds-db:connect. I manually created this IAM user into PG database, but without password and attached rds_iam role. However, I tried various ways to authenticate with the Proxy using IAM, but nothing seems to be working. I would really appreciate, if someone can please provide the code sample for this in Python. I am not even sure which certificate to use. Please help.lg...

I am using an Athena table sourced from parquet files in s3. The table consists mostly of columns of type array<double>. To create the files, I am converting pandas dataframes into the parquet files. When I try to query the table, I am receiving a HIVE error that states: "HIVE_CANNOT_OPEN_SPLIT: Error opening Hive split s3://<bucket>/<key>.parquet (offset=0, length=1700612): org.apache.parquet.io.PrimitiveColumnIO cannot be cast to org.apache.parquet.io.GroupColumnIO" I have narrowed down the issue to being one of the array<double> column that includes Python None values. I believe Athena is trying to convert those empty values to an array, and that is where the issue arises. I am able to resolve this by using empty an empty list [] instead of None in the pandas column. However, we would prefer for these values to show up as null in Athena rather than as an empty list. Is there any way to have the empty values work with Athena while being null rather than an empty list? Thank youlg...

I have SQL Server 2016 running on Windows Server 2016 with 4 CPU and 16 GB RAM. We are migrating SQL Server to AWS RDS SQL Server. I think RDS runs only SQL Server engine and it is lightweight than Windows Server which runs various services required for OS. So I think while choosing RDS instance type e.g. xlarge, 2xlarge etc. we should not look for exactly same CPU and RAM for RDS instance. I guess RDS large which has 2 CPU and 8 GB RAM will match the SQL server running on Windows server with 4 CPU and 16 GB?lg...

I'm new to using RDS, and I'm concerned about backing up MySQL DB on RDS . so i made 1 manual snapshot and when i do need to restore my DB via snapshot , does the transaction data stored in the tables also restored?lg...

Hi, I would like to update my current Postgres database from version 9.6.24 to 10.19. When update is done, I see precheck log file that indicates migration failure because of Postgis extension. Current POstgis extension is 2.5.5 in my 9.6.24 Postgres database. Do you have any idea how to manage this issue ? I use this extension in my database so I can't delete this extension ( because has I understand this will delete associated column in my database). Thanks in advance for your help. Best Regardslg...

We have changed the RDS postgres parameters by changing the log_statement in the aws RDS console. However it is not reflecting in pGAdmin. Can we assume that the log changes would be reflected back in pGAdmin(audit log)lg...

1. Please tell me about SSL connection to RDS. In a case where a Go application in EC2 (Amazon Linux) accesses a database via SSL to RDS (PostgreSQL) in the same region I am planning to follow the procedure below, but I would like to confirm something. https://docs.aws.amazon.com/ja_jp/AmazonRDS/latest/UserGuide/PostgreSQL.Concepts.General.SSL.html#PostgreSQL.Concepts.General.SSL.Connecting In the "Connecting to a PostgreSQL DB instance via SSL" step "Import the certificate into the operating system. In the "Connecting to a PostgreSQL DB instance via SSL" section, it states that the certificate needs to be imported into the trust store. In fact, by specifying sslrootcert and sslmode in the psql command without this step, the SSL connection status (ssl_is_used) returns "t" and it looks like an SSL connection is established. I would like to know if this procedure is necessary for this connection from the Go application. 2. How does it work? For the public key (rds-ca-2019) specified in the sslrootcert I assume that a common key is required for a normal SSL connection, but is this generated by RDS or a driver such as libpq? Also, is it correct that the server certificate and private key are installed in the DB when the RDS (PostgreSQL) is created?lg...

I want to enter into digital game distribution services, similar to STEAM. I am new to all this. Need help and suggestions on how can I develop a steam-like platform where 1. There will be 2 interfaces - 1 for gamers, 1 for the game developer 2. Game codes to be stored in the cloud 3. User authentication will be needed 4. Game license management will be needed Thankslg...

We want to have a Lambda point at a small, yet important DynamoDB table that has Point-in-time recovery enabled. Running through the recovery process, we see it restores to a new table as opposed to restoring the original table in place. With it restoring to a new table, is the idea that one would typically change the Lambda to point at the new table, or would one copy the new table over the original? Related: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/PointInTimeRecovery.htmllg...

Hello, I'm trying to migrate an SQL Server database to RDS which uses the SQL Server agent and has a store procedure that selects from the SQL Server Agent Table dbo.sysjobsteps as per https://learn.microsoft.com/en-us/sql/relational-databases/system-tables/dbo-sysjobsteps-transact-sql?view=sql-server-ver15. I've tried assigning permissions as per https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Appendix.SQLServer.CommonDBATasks.Agent.html but I keep getting the error: The SELECT permission was denied on the object 'sysjobsteps', database 'msdb', schema 'dbo'. (even with the master user). On the other hand, other tables like dbo.sysjobs do return the expected results. Is this expected and does anyone know a workaround?lg...