CloudFormation RDS CreateInstance fails incompatible-parameters

0

We have been creating RDS MariaDB instances on an almost daily basis using CloudFormation and our scripts for a long time (years), however, the last week or two RDS CreateInstance fails intermittently with the below error - deleting the stack and retrying usually works, but of course not a suitable long term solution for regular environment creation:

2022-04-14 15:26:20 UTC+0100	RdsInstance	CREATE_FAILED	DB Instance is in state: incompatible-parameters

If I view the RDS database in question, under the "Events" listing there are about 5 pages of the same event:

April 14, 2022, 3:10:13 PM UTC
Your MySql memory setting is inappropriate for the usages

On that failure, it attempts to roll back, but that also fails because the delete DB instance fails for the same reason (more-or-less, DB not in available state).

However, the DB does eventually end up being available, after something like 20 mins or so, the DB (having failed to be deleted) will be showing as "Available". We have not changed anything in the parameter group or DB engine. It is running MariaDB 10.3.31.

Anyone have any idea what might be causing this or what might have changed recently?


EDIT: Following on from the answers provided so far, the thing I'm most interested in is the intermittent nature of the issue, and the fact that its just started happening, having run successfully for a long time previously. If there was an incorrect parameter for the DB type, then I'd expect it to fail every time, the intermittent nature of it makes me thinking its more likely a race condition or timing issue involved.

I have reviewed the parameter group, and only one value has been changed from the default params for Mariadb10.3: max_connections - now set to a fixed 1000 (rather than the default to calculate it based on the size of the instance). This hasn't changed for a long time, and can't see that this is causing an issue?

2 Answers
0

That error is telling you that you have a value set in your custom parameter group that is preventing the database from starting up. There is probably a memory value that is set to high for your selected instance type. You can review the parameters or try again with the default parameter group.

AWS
MODERATOR
philaws
answered 2 years ago
  • Thanks, will review the params - any idea why it would suddenly start happening intermittently? I would have thought if it was a config error it would happen every time (often the DB does start without issue)? We have been running the process without changing param group/engine for a long time now too.

0

Hello , This is more likely seems to be an issue with the parameter (memory) in the parameter group . Not sure what exact parameters are being used and without looking more into the logs cannot pinpoint exact issue but to isolate if parameters that are currently configured is the issue, can you use Default parameter group in CFN and try to create RDS multiple times and see if it gets spun up without any issue ? I am assuming you are using Custom Parameter group and not default one for your RDS spin up.

Please take a look if you haven't the below link for additional info

https://aws.amazon.com/premiumsupport/knowledge-center/rds-incompatible-parameters/

AWS
answered 2 years ago
  • thanks - I'm sure the default param group will resolve it, but what I'm really more interested in is:

    1. Why after working consistently reliable (multiple DBs every day, for probably years) it started failing now
    2. Why it fails sometimes, but still works sometimes

    Basically, I'd have assumed if a parameter is wrong, it would just be binary and not work? not sometimes work and sometimes fail

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions