feat(rds): add support for manageMasterUserPassword in L2 construct #30997

epoctic · 2024-08-01T01:29:01Z

Issue # (if applicable)

Reason for this change

In order to make the properties in the manageMasterUserPassword of the L1 Construct CfnDBCluster configurable in the L2 Construct.

Even when using a CFN override to set manageMasterUserPassword, the L2 cluster construct would create an additional unused secret. This PR adds native support for the manageMasterUserPassword CloudFormation attribute and eliminates the extra secret that the workaround caused.

Description of changes

To ensure existing clusters wouldn't have a new cfn property added and set to false, I had to use a few conditional statements. I also wanted to make sure that the created secret supported a customer defined username and kms key.

Description of how you validated changes

I deployed a database cluster using the new code and logged into it using the managed password with custom username. I also manually triggered rotations on the secret and ensured that the newly generated secret also worked for logging into the database.

Checklist

[ x] My code adheres to the CONTRIBUTING GUIDE and DESIGN GUIDELINES

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

lpizzinidev

Thanks 👍 Left some comments for adjustments.
Also, can you please add unit test coverage?

packages/aws-cdk-lib/aws-rds/lib/cluster.ts

packages/aws-cdk-lib/aws-rds/README.md

epoctic · 2024-08-06T00:00:48Z

Thanks 👍 Left some comments for adjustments. Also, can you please add unit test coverage?

Done, sorry about the wait!

packages/aws-cdk-lib/aws-rds/lib/cluster.ts

lpizzinidev · 2024-08-07T15:47:44Z

packages/aws-cdk-lib/aws-rds/test/cluster.test.ts

+
+    });
+
+    test('throw error for setting `manageMasterUserPassword` to true while `credentials` props excludeCharacters is defined', () => {


We could unify these test cases:

test.each([ ['excludeCharacters', { excludeCharacters: '1234' }], ['secret', { secret: new sm.Secret(stack, 'secret') }], ])('throw error for setting `manageMasterUserPassword` to true while `credentials.%s` is defined', (_, credentials) => { ... }

We could unify these test cases:

test.each([ ['excludeCharacters', { excludeCharacters: '1234' }], ['secret', { secret: new sm.Secret(stack, 'secret') }], ])('throw error for setting `manageMasterUserPassword` to true while `credentials.%s` is defined', (_, credentials) => { ... }

Sorry about the wait! It took a bit to work through how I could define an external stack and still keep resources unique.

epoctic · 2024-08-13T20:39:38Z

AWS CodeBuild CI Report

CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv

Commit ID: 02cebb7

Result: FAILED

Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

This failure doesn't seem to be related to code I've committed.

Hawxy · 2024-09-10T08:14:07Z

packages/aws-cdk-lib/aws-rds/lib/cluster.ts

+        masterUsername: props.credentials?.username ?? Credentials.fromUsername(props.engine.defaultUsername ?? 'admin').username,
+        // Define kms key the RDS Managed Secret will use.
+        ...props.credentials?.encryptionKey !== undefined && {
+          masterUserSecret: {


Shouldn't this.secret be set by CfnDBCluster's masterUserSecret.secretArn? I don't see a direct way to access the created secret for use within CDK.

There wouldn't be one unfortunately. I'm not sure Cfn has a way to reference it either. The AWS SDK does allow you to get the secret's ARN by describing the cluster.

Not sure if it helps, but this is how I was getting access to the secret so that I could include it in the task definition for my ECS tasks that are created by the same stack.

((CfnDBCluster)cfnCluster.Node.DefaultChild).AttrMasterUserSecretSecretArn

@antonfelich Thanks for the comment! After a bit of testing I was able to use a similar technique to create an imported secret that's accessible from the cluster's secret prop. Should be a great addition!

Line 1169 in cluster.ts:
this.secret = secretsmanager.Secret.fromSecretCompleteArn(this, 'DbClusterSecret', cluster.attrMasterUserSecretSecretArn)

aws-cdk-automation · 2024-09-29T00:12:20Z

This PR has been in the MERGE CONFLICTS state for 3 weeks, and looks abandoned. To keep this PR from being closed, please continue work on it. If not, it will automatically be closed in a week.

antonfelich · 2024-10-02T06:38:45Z

Any idea when this might be merged in? I've been using the CfnDBCluster ManageMasterUserPassword work around, but suddenly today it's decided not to work any more. The unused secret seems to be attaching to the cluster first, and then when the real secret is trying to attach it blows up because there already is one. CloudFormation just gives a Internal Failure (RequestToken: 6e94374a-4094-6627-e639-c8c95a165caf) which isn't particularly helpful unfortunately.

epoctic · 2024-10-02T15:01:26Z

Any idea when this might be merged in? I've been using the CfnDBCluster ManageMasterUserPassword work around, but suddenly today it's decided not to work any more. The unused secret seems to be attaching to the cluster first, and then when the real secret is trying to attach it blows up because there already is one. CloudFormation just gives a Internal Failure (RequestToken: 6e94374a-4094-6627-e639-c8c95a165caf) which isn't particularly helpful unfortunately.

@antonfelich I'm still waiting on a community approval, please feel free to review it!

antonfelich

Looks good to me, keen to see this out in the wild!

antonfelich · 2024-10-02T22:54:29Z

packages/aws-cdk-lib/aws-rds/lib/cluster.ts

+    if (
+      props.manageMasterUserPassword &&
+      (
+        props.credentials?.excludeCharacters ||


It might already be taken care of further up, but ManageMasterUserPassword and MasterUserPassword are mutually exclusive, so would be good to throw if they accidentally set both.

When I was manually setting ManageMasterUserPassword I had to intentionally remove the MasterUserPassword attribute.

var cfnCluster = (CfnDBCluster)dbCluster.Node.DefaultChild; cfnCluster.AddPropertyOverride("ManageMasterUserPassword", true); cfnCluster.AddPropertyDeletionOverride("MasterUserPassword");

Sorry almost forgot, because one of my clusters already existed and I was migrating it from manual password to RDS managed, I also had to blank out the username.

cfnCluster.AddPropertyDeletionOverride("MasterUsername");

antonfelich · 2024-10-02T22:56:37Z

packages/aws-cdk-lib/aws-rds/lib/cluster.ts

+        masterUsername: props.credentials?.username ?? Credentials.fromUsername(props.engine.defaultUsername ?? 'admin').username,
+        // Define kms key the RDS Managed Secret will use.
+        ...props.credentials?.encryptionKey !== undefined && {
+          masterUserSecret: {


Not sure if it helps, but this is how I was getting access to the secret so that I could include it in the task definition for my ECS tasks that are created by the same stack.

((CfnDBCluster)cfnCluster.Node.DefaultChild).AttrMasterUserSecretSecretArn

aws-cdk-automation · 2024-10-04T00:10:49Z

This PR cannot be merged because it has conflicts. Please resolve them. The PR will be considered stale and closed if it remains in an unmergeable state.

aws-cdk-automation · 2024-10-05T00:11:18Z

This PR cannot be merged because it has conflicts. Please resolve them. The PR will be considered stale and closed if it remains in an unmergeable state.

aws-cdk-automation · 2024-10-06T00:12:01Z

This PR cannot be merged because it has conflicts. Please resolve them. The PR will be considered stale and closed if it remains in an unmergeable state.

…truct

Co-authored-by: Luca Pizzini <lpizzini7@gmail.com>

aws-cdk-automation · 2024-10-11T14:46:53Z

AWS CodeBuild CI Report

CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
Commit ID: d589245
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

github-actions bot added beginning-contributor [Pilot] contributed between 0-2 PRs to the CDK effort/medium Medium work item – several days of effort feature-request A feature should be added or improved. p2 labels Aug 1, 2024

aws-cdk-automation requested a review from a team August 1, 2024 01:29

epoctic force-pushed the epoctic/support-rds-managed-cluster-secret branch from 7ba289f to de886c9 Compare August 1, 2024 01:36

epoctic mentioned this pull request Aug 1, 2024

AWS::RDS::DBCluster Password management with Amazon Aurora and AWS Secrets Manager #29239

Open

2 tasks

aws-cdk-automation added the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label Aug 1, 2024

lpizzinidev suggested changes Aug 1, 2024

View reviewed changes

packages/aws-cdk-lib/aws-rds/lib/cluster.ts Outdated Show resolved Hide resolved

packages/aws-cdk-lib/aws-rds/lib/cluster.ts Show resolved Hide resolved

packages/aws-cdk-lib/aws-rds/README.md Outdated Show resolved Hide resolved

packages/aws-cdk-lib/aws-rds/README.md Show resolved Hide resolved

aws-cdk-automation removed the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label Aug 1, 2024

github-actions bot mentioned this pull request Aug 5, 2024

Weekly PR metrics report #31022

Closed

epoctic force-pushed the epoctic/support-rds-managed-cluster-secret branch from 0bd1d1d to 3909d7e Compare August 5, 2024 22:19

epoctic changed the title ~~feat(rds): add support for cluster managed master password in L2 cons…~~ feat(rds): add support for manageMasterUserPassword in L2 construct Aug 5, 2024

epoctic requested a review from lpizzinidev August 6, 2024 00:04

epoctic force-pushed the epoctic/support-rds-managed-cluster-secret branch from 58f8a79 to 5f89799 Compare August 6, 2024 17:58

lpizzinidev suggested changes Aug 7, 2024

View reviewed changes

epoctic requested a review from lpizzinidev August 13, 2024 00:51

epoctic force-pushed the epoctic/support-rds-managed-cluster-secret branch 2 times, most recently from c47961e to 02cebb7 Compare August 13, 2024 20:20

epoctic force-pushed the epoctic/support-rds-managed-cluster-secret branch from 02cebb7 to 744e065 Compare August 14, 2024 00:38

github-actions bot mentioned this pull request Sep 1, 2024

Monthly PR metrics report #31279

Open

epoctic requested a review from a-bigelow September 5, 2024 13:29

Hawxy reviewed Sep 10, 2024

View reviewed changes

epoctic force-pushed the epoctic/support-rds-managed-cluster-secret branch from 744e065 to 7b0ecee Compare September 30, 2024 19:43

epoctic requested a review from Hawxy October 2, 2024 15:02

antonfelich approved these changes Oct 2, 2024

View reviewed changes

epoctic force-pushed the epoctic/support-rds-managed-cluster-secret branch 6 times, most recently from e1d6ca7 to c646a07 Compare October 10, 2024 23:34

Jonathan Colbert and others added 7 commits October 11, 2024 09:52

feat(rds): add support for cluster managed master password in L2 cons…

c313d57

…truct

remove unnecessary spread operation

0fde1ba

Co-authored-by: Luca Pizzini <lpizzini7@gmail.com>

add sources for RDS and Secrets Manager integrations

d9f03c9

Co-authored-by: Luca Pizzini <lpizzini7@gmail.com>

add check to prevent defining conflicting properties

1d3a4c2

add manageMasterUserPassword integration tests

1875775

combine toThrow unit tests into test loop

3ddee01

set cluster secret when using manageMasterUserPassword

d589245

epoctic force-pushed the epoctic/support-rds-managed-cluster-secret branch 2 times, most recently from d137d34 to d589245 Compare October 11, 2024 14:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(rds): add support for manageMasterUserPassword in L2 construct #30997

feat(rds): add support for manageMasterUserPassword in L2 construct #30997

epoctic commented Aug 1, 2024 •

edited

Loading

lpizzinidev left a comment

epoctic commented Aug 6, 2024 •

edited

Loading

lpizzinidev Aug 7, 2024 •

edited

Loading

epoctic Aug 13, 2024

epoctic commented Aug 13, 2024

AWS CodeBuild CI Report

Hawxy Sep 10, 2024

epoctic Sep 30, 2024

antonfelich Oct 2, 2024

epoctic Oct 6, 2024

aws-cdk-automation commented Sep 29, 2024

antonfelich commented Oct 2, 2024

epoctic commented Oct 2, 2024 •

edited

Loading

antonfelich left a comment

antonfelich Oct 2, 2024

antonfelich Oct 2, 2024

antonfelich Oct 2, 2024

aws-cdk-automation commented Oct 4, 2024

aws-cdk-automation commented Oct 5, 2024

aws-cdk-automation commented Oct 6, 2024

aws-cdk-automation commented Oct 11, 2024


		});

		test('throw error for setting `manageMasterUserPassword` to true while `credentials` props excludeCharacters is defined', () => {

feat(rds): add support for manageMasterUserPassword in L2 construct #30997

Are you sure you want to change the base?

feat(rds): add support for manageMasterUserPassword in L2 construct #30997

Conversation

epoctic commented Aug 1, 2024 • edited Loading

Issue # (if applicable)

Reason for this change

Description of changes

Description of how you validated changes

Checklist

lpizzinidev left a comment

Choose a reason for hiding this comment

epoctic commented Aug 6, 2024 • edited Loading

lpizzinidev Aug 7, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

epoctic commented Aug 13, 2024

AWS CodeBuild CI Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aws-cdk-automation commented Sep 29, 2024

antonfelich commented Oct 2, 2024

epoctic commented Oct 2, 2024 • edited Loading

antonfelich left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aws-cdk-automation commented Oct 4, 2024

aws-cdk-automation commented Oct 5, 2024

aws-cdk-automation commented Oct 6, 2024

aws-cdk-automation commented Oct 11, 2024

AWS CodeBuild CI Report

epoctic commented Aug 1, 2024 •

edited

Loading

epoctic commented Aug 6, 2024 •

edited

Loading

lpizzinidev Aug 7, 2024 •

edited

Loading

epoctic commented Oct 2, 2024 •

edited

Loading