Describe the bug

When using CDK to set up a Secrets Manager RotationSchedule, creation of the AWS::SecretsManager::RotationSchedule resource can fail (in a small proportion of deployments) with the following error:

Secrets Manager cannot invoke the specified Lambda function. Ensure that the function policy grants access to the principal secretsmanager.amazonaws.com. (Service: AWSSecretsManager; Status Code: 400; Error Code: AccessDeniedException

From looking at the Stack events, I can see that the Lambda Invoke permission and the RotationSchedule are being created at the same time, so it's down to luck whether the RotationSchedule's test invoke succeeds or fails, though in most cases it seems to succeed. An explicit dependency to ensure the Lambda resource policy is created first would improve the deployment reliability.

Expected Behavior

The deployment succeeds reliably.

Current Behavior

The deployment failed with the above error in 2 out of 18 deployments.

Reproduction Steps

Create a new typescript CDK project:

mkdir cdk-bug; cd cdk-bug
cdk init --language=typescript

Replace lib/cdk-bug-stack.ts with the following:

import * as cdk from 'aws-cdk-lib';
import * as lambda from 'aws-cdk-lib/aws-lambda';
import { Secret } from 'aws-cdk-lib/aws-secretsmanager';
import { Construct } from 'constructs';

export class CdkBugStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const secret = new Secret(this, "Secret", {});
    const rotationLambda = new lambda.Function(this, "RotationLambda", {
      runtime: lambda.Runtime.NODEJS_18_X,
      handler: 'index.handler',
      code: lambda.Code.fromInline('exports.handler = async () => { console.log("Fake rotation lambda"); };'),
    });

    secret.addRotationSchedule("RotationSchedule", {
      rotationLambda,
    });
  }
}

A cdk synth shows that the SecretRotationSchedule49AED07D resource has no dependency on the AWS::Lambda::Permission resource RotationLambdaInvokeN0a2GKfZP0JmDqDEVhhu6A0TUv3NyNbk4YMFKNc852E0E9A.

Possible Solution

Add an explicit dependency to ensure the Lambda resource policy is created before the RotationSchedule.

Additional Information/Context

No response

CDK CLI Version

2.88.0 (build 5d497f9)

Framework Version

2.87.0

Node.js Version

v18.17.0

OS

MacOS 13.4.1 (c) (22F770820d)

Language

Typescript

Language Version

TypeScript (5.1.6)

Other information

No response