fix: UserPool, Volume, ElasticSearch, FSx are now RETAIN by default (#…

…12920)

(Gave this change an eye-catching title so it would stand out in the change log)

`cfn-lint` expects all stateful resources to have a removal policy, but we weren't providing that option at the L2 level yet, and the L1 API is cumbersome.

Add the `removalPolicy` option to the following resources:

* Cognito User Pools (default: RETAIN)
* EC2 Volume (default: RETAIN)
* ElasticSearch Domain (default: RETAIN)
* FSx FileSystem (default: RETAIN)
* SQS Queue (default: DESTROY)
* Nested Stack (default: DESTROY)

All L2 resources now have an `applyRemovalPolicy` method, so it can be set even for non-stateful resources.

A mechanism has been added to the codegen so that new L2 authors cannot forget about the `removalPolicy` when they're writing an L2.

I'm aware that the choice to make most of these RETAIN by default is going to be contentious. There are 2 questions here:

* Is RETAIN the correct default in general?
* Can we afford switching from implicit-DESTROY to implicit-RETAIN?

### Is RETAIN the correct default?

I would argue "yes", by process of elimination:

Defaulting `removalPolicy` for all resources to `DESTROY` if nothing is given (and writing `DESTROY` to the template) is just going to put us back to the situation in CloudFormation *before* `cfn-lint` existed (your stateful resources are going to be destroyed and nothing is going to warn you about it).

Making `removalPolicy` explicitly required for all stateful resources is a backwards-breaking change, and the point about CDK is that it will have sane defaults.

I can't come up with a better solution than picking RETAIN as default. 

### Can we afford changing the default here?

I'm sensitive to the arguments that this a breaking change which we can't afford. 

It feels like this is the value we would have given the resources had we thought about this earlier, and not being able to correct past mistakes by saying they are "locked in" is going to be a death knell for the project. 

We could go for a "future behavior" flag, but the risk here seems minimal: this is a change that REDUCES risk of data loss. It might increase risk of deployment errors (duplicate resource names), but that can be manually managed. I will fast-follow with a change that will introduce warnings for resources that have a physical name set and also a RETAIN policy, to clearly identify you're doing something dangerous in CloudFormation. I'm not sure it's worth the effort to go further than that.

Although please: discussion welcome.

Fixes #12563.

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*

packages/@aws-cdk-containers/ecs-service-extensions/test/integ.assign-public-ip.expected.json

@@ -400,7 +400,9 @@
       "Type": "AWS::SQS::Queue",
       "Properties": {
         "MessageRetentionPeriod": 1209600
-      }
+      },
+      "UpdateReplacePolicy": "Delete",
+      "DeletionPolicy": "Delete"
     },
     "nameserviceTaskRecordManagerEventsQueueF805A6C1": {
       "Type": "AWS::SQS::Queue",
@@ -415,7 +417,9 @@
           "maxReceiveCount": 500
         },
         "VisibilityTimeout": 30
-      }
+      },
+      "UpdateReplacePolicy": "Delete",
+      "DeletionPolicy": "Delete"
     },
     "nameserviceTaskRecordManagerEventsQueuePolicy65CC6F9E": {
       "Type": "AWS::SQS::QueuePolicy",
@@ -746,14 +750,12 @@
             ]
           }
         },
-        "Handler": "index.queue_handler",
         "Role": {
           "Fn::GetAtt": [
             "nameserviceTaskRecordManagerEventHandlerServiceRoleE66EE52A",
             "Arn"
           ]
         },
-        "Runtime": "python3.8",
         "Environment": {
           "Variables": {
             "HOSTED_ZONE_ID": {
@@ -777,7 +779,9 @@
             }
           }
         },
+        "Handler": "index.queue_handler",
         "ReservedConcurrentExecutions": 1,
+        "Runtime": "python3.8",
         "Timeout": 30
       },
       "DependsOn": [
@@ -788,14 +792,14 @@
     "nameserviceTaskRecordManagerEventHandlerSqsEventSourceawsecsintegnameserviceTaskRecordManagerEventsQueueC5EE9A869F1EB155": {
       "Type": "AWS::Lambda::EventSourceMapping",
       "Properties": {
+        "FunctionName": {
+          "Ref": "nameserviceTaskRecordManagerEventHandler4B8C6905"
+        },
         "EventSourceArn": {
           "Fn::GetAtt": [
             "nameserviceTaskRecordManagerEventsQueueF805A6C1",
             "Arn"
           ]
-        },
-        "FunctionName": {
-          "Ref": "nameserviceTaskRecordManagerEventHandler4B8C6905"
         }
       }
     },
@@ -909,13 +913,13 @@
             ]
           }
         },
-        "Handler": "index.cleanup_resource_handler",
         "Role": {
           "Fn::GetAtt": [
             "nameserviceTaskRecordManagerCleanupResourceProviderHandlerServiceRoleCCA462F0",
             "Arn"
           ]
         },
+        "Handler": "index.cleanup_resource_handler",
         "Runtime": "python3.8",
         "Timeout": 300
       },
@@ -1022,14 +1026,12 @@
             ]
           }
         },
-        "Handler": "framework.onEvent",
         "Role": {
           "Fn::GetAtt": [
             "nameserviceTaskRecordManagerCleanupResourceProviderframeworkonEventServiceRoleF0570BD0",
             "Arn"
           ]
         },
-        "Runtime": "nodejs10.x",
         "Description": "AWS CDK resource provider framework - onEvent (aws-ecs-integ/name-service/TaskRecordManager/CleanupResourceProvider)",
         "Environment": {
           "Variables": {
@@ -1041,6 +1043,8 @@
             }
           }
         },
+        "Handler": "framework.onEvent",
+        "Runtime": "nodejs10.x",
         "Timeout": 900
       },
       "DependsOn": [
@@ -1242,13 +1246,13 @@
             ]
           }
         },
-        "Handler": "index.handler",
         "Role": {
           "Fn::GetAtt": [
             "AWS679f53fac002430cb0da5b7982bd2287ServiceRoleC1EA0FF2",
             "Arn"
           ]
         },
+        "Handler": "index.handler",
         "Runtime": "nodejs12.x",
         "Timeout": 120
       },

packages/@aws-cdk-containers/ecs-service-extensions/test/integ.imported-environment.expected.json

@@ -49,7 +49,9 @@
             ]
           ]
         }
-      }
+      },
+      "UpdateReplacePolicy": "Delete",
+      "DeletionPolicy": "Delete"
     },
     "ServiceloadbalancerD5D60894": {
       "Type": "AWS::ElasticLoadBalancingV2::LoadBalancer",

packages/@aws-cdk/aws-apigateway/test/authorizers/integ.cognito-authorizer.expected.json

@@ -27,7 +27,9 @@
           "EmailSubject": "Verify your new account",
           "SmsMessage": "The verification code to your new account is {####}"
         }
-      }
+      },
+      "UpdateReplacePolicy": "Retain",
+      "DeletionPolicy": "Retain"
     },
     "myauthorizer23CB99DD": {
       "Type": "AWS::ApiGateway::Authorizer",

packages/@aws-cdk/aws-apigateway/test/integ.restapi-import.lit.expected.json

@@ -130,7 +130,9 @@
             "Ref": "RestApi0C43BF4B"
           }
         }
-      }
+      },
+      "UpdateReplacePolicy": "Delete",
+      "DeletionPolicy": "Delete"
     },
     "integrestapiimportBooksStackNestedStackintegrestapiimportBooksStackNestedStackResource395C2C9B": {
       "Type": "AWS::CloudFormation::Stack",
@@ -192,7 +194,9 @@
             "Ref": "RestApi0C43BF4B"
           }
         }
-      }
+      },
+      "UpdateReplacePolicy": "Delete",
+      "DeletionPolicy": "Delete"
     },
     "integrestapiimportDeployStackNestedStackintegrestapiimportDeployStackNestedStackResource0D0EE737": {
       "Type": "AWS::CloudFormation::Stack",
@@ -252,7 +256,9 @@
       "DependsOn": [
         "integrestapiimportBooksStackNestedStackintegrestapiimportBooksStackNestedStackResource395C2C9B",
         "integrestapiimportPetsStackNestedStackintegrestapiimportPetsStackNestedStackResource2B31898B"
-      ]
+      ],
+      "UpdateReplacePolicy": "Delete",
+      "DeletionPolicy": "Delete"
     }
   },
   "Outputs": {

packages/@aws-cdk/aws-apigatewayv2-authorizers/test/http/integ.user-pool.expected.json

@@ -157,7 +157,9 @@
           "EmailSubject": "Verify your new account",
           "SmsMessage": "The verification code to your new account is {####}"
         }
-      }
+      },
+      "UpdateReplacePolicy": "Retain",
+      "DeletionPolicy": "Retain"
     },
     "userpoolmyclientFAD947AB": {
       "Type": "AWS::Cognito::UserPoolClient",

packages/@aws-cdk/aws-appsync/test/integ.graphql-iam.expected.json

@@ -28,7 +28,9 @@
           "EmailSubject": "Verify your new account",
           "SmsMessage": "The verification code to your new account is {####}"
         }
-      }
+      },
+      "UpdateReplacePolicy": "Retain",
+      "DeletionPolicy": "Retain"
     },
     "ApiF70053CD": {
       "Type": "AWS::AppSync::GraphQLApi",

packages/@aws-cdk/aws-appsync/test/integ.graphql.expected.json

@@ -28,7 +28,9 @@
           "EmailSubject": "Verify your new account",
           "SmsMessage": "The verification code to your new account is {####}"
         }
-      }
+      },
+      "UpdateReplacePolicy": "Retain",
+      "DeletionPolicy": "Retain"
     },
     "ApiF70053CD": {
       "Type": "AWS::AppSync::GraphQLApi",

packages/@aws-cdk/aws-backup/test/integ.backup.expected.json

@@ -24,7 +24,9 @@
       "DeletionPolicy": "Delete"
     },
     "FileSystem": {
-      "Type": "AWS::EFS::FileSystem"
+      "Type": "AWS::EFS::FileSystem",
+      "UpdateReplacePolicy": "Delete",
+      "DeletionPolicy": "Delete"
     },
     "Vault23237E5B": {
       "Type": "AWS::Backup::BackupVault",

packages/@aws-cdk/aws-backup/test/integ.backup.ts

@@ -16,7 +16,8 @@ class TestStack extends Stack {
       removalPolicy: RemovalPolicy.DESTROY,
     });
 
-    new efs.CfnFileSystem(this, 'FileSystem');
+    const fs = new efs.CfnFileSystem(this, 'FileSystem');
+    fs.applyRemovalPolicy(RemovalPolicy.DESTROY);
 
     const vault = new backup.BackupVault(this, 'Vault', {
       removalPolicy: RemovalPolicy.DESTROY,

packages/@aws-cdk/aws-backup/test/selection.test.ts

@@ -2,7 +2,7 @@ import '@aws-cdk/assert/jest';
 import * as dynamodb from '@aws-cdk/aws-dynamodb';
 import * as ec2 from '@aws-cdk/aws-ec2';
 import * as efs from '@aws-cdk/aws-efs';
-import { Stack } from '@aws-cdk/core';
+import { RemovalPolicy, Stack } from '@aws-cdk/core';
 import { Construct } from 'constructs';
 import { BackupPlan, BackupResource, BackupSelection } from '../lib';
 
@@ -128,7 +128,8 @@ test('fromConstruct', () => {
   class EfsConstruct extends CoreConstruct {
     constructor(scope: Construct, id: string) {
       super(scope, id);
-      new efs.CfnFileSystem(this, 'FileSystem');
+      const fs = new efs.CfnFileSystem(this, 'FileSystem');
+      fs.applyRemovalPolicy(RemovalPolicy.DESTROY);
     }
   }
   class MyConstruct extends CoreConstruct {

packages/@aws-cdk/aws-cloudformation/test/integ.nested-stack.expected.json

@@ -1,7 +1,9 @@
 {
   "Resources": {
     "SubscriberQueueC193DC66": {
-      "Type": "AWS::SQS::Queue"
+      "Type": "AWS::SQS::Queue",
+      "UpdateReplacePolicy": "Delete",
+      "DeletionPolicy": "Delete"
     },
     "SubscriberQueuePolicy25A0799E": {
       "Type": "AWS::SQS::QueuePolicy",
@@ -196,7 +198,9 @@
             "Ref": "SubscriberQueueC193DC66"
           }
         }
-      }
+      },
+      "UpdateReplacePolicy": "Delete",
+      "DeletionPolicy": "Delete"
     },
     "NestedStack2NestedStackNestedStack2NestedStackResourceFDF82E43": {
       "Type": "AWS::CloudFormation::Stack",
@@ -250,7 +254,9 @@
         "Parameters": {
           "TopicNamePrefix": "Prefix2"
         }
-      }
+      },
+      "UpdateReplacePolicy": "Delete",
+      "DeletionPolicy": "Delete"
     }
   },
   "Parameters": {

packages/@aws-cdk/aws-cloudformation/test/integ.nested-stacks-assets.expected.json

@@ -17,7 +17,7 @@
               },
               "/",
               {
-                "Ref": "AssetParameters28e3582cfc7c551f42435f44110c499fb2c415fe0f02a02a77139cf43edd1145S3BucketFDBB032A"
+                "Ref": "AssetParameters7bd7ccfde94a4ad94f13971f10733ee01020eb6d7538b7a5c2b05966db2c0aabS3Bucket2E4E0318"
               },
               "/",
               {
@@ -27,7 +27,7 @@
                     "Fn::Split": [
                       "||",
                       {
-                        "Ref": "AssetParameters28e3582cfc7c551f42435f44110c499fb2c415fe0f02a02a77139cf43edd1145S3VersionKey58263016"
+                        "Ref": "AssetParameters7bd7ccfde94a4ad94f13971f10733ee01020eb6d7538b7a5c2b05966db2c0aabS3VersionKeyF6AB1DF2"
                       }
                     ]
                   }
@@ -40,7 +40,7 @@
                     "Fn::Split": [
                       "||",
                       {
-                        "Ref": "AssetParameters28e3582cfc7c551f42435f44110c499fb2c415fe0f02a02a77139cf43edd1145S3VersionKey58263016"
+                        "Ref": "AssetParameters7bd7ccfde94a4ad94f13971f10733ee01020eb6d7538b7a5c2b05966db2c0aabS3VersionKeyF6AB1DF2"
                       }
                     ]
                   }
@@ -50,40 +50,42 @@
           ]
         },
         "Parameters": {
-          "referencetonestedstacksassetsAssetParametersb13aad60258df1fbe5fb1312a7b2f8f25c03b3e07113782f7c12f00e023e148aS3BucketE2268D38Ref": {
-            "Ref": "AssetParametersb13aad60258df1fbe5fb1312a7b2f8f25c03b3e07113782f7c12f00e023e148aS3BucketC5E2D427"
+          "referencetonestedstacksassetsAssetParametersbbe209afddb09a12327bab7a105e085758a29b769b5b4bf5b6320ac41b05fc51S3BucketFE27EEBCRef": {
+            "Ref": "AssetParametersbbe209afddb09a12327bab7a105e085758a29b769b5b4bf5b6320ac41b05fc51S3Bucket8C5997AB"
           },
-          "referencetonestedstacksassetsAssetParametersb13aad60258df1fbe5fb1312a7b2f8f25c03b3e07113782f7c12f00e023e148aS3VersionKeyD31C6796Ref": {
-            "Ref": "AssetParametersb13aad60258df1fbe5fb1312a7b2f8f25c03b3e07113782f7c12f00e023e148aS3VersionKey31422F11"
+          "referencetonestedstacksassetsAssetParametersbbe209afddb09a12327bab7a105e085758a29b769b5b4bf5b6320ac41b05fc51S3VersionKey24D35F02Ref": {
+            "Ref": "AssetParametersbbe209afddb09a12327bab7a105e085758a29b769b5b4bf5b6320ac41b05fc51S3VersionKey81BEC7FB"
           }
         }
-      }
+      },
+      "UpdateReplacePolicy": "Delete",
+      "DeletionPolicy": "Delete"
     }
   },
   "Parameters": {
-    "AssetParametersb13aad60258df1fbe5fb1312a7b2f8f25c03b3e07113782f7c12f00e023e148aS3BucketC5E2D427": {
+    "AssetParametersbbe209afddb09a12327bab7a105e085758a29b769b5b4bf5b6320ac41b05fc51S3Bucket8C5997AB": {
       "Type": "String",
-      "Description": "S3 bucket for asset \"b13aad60258df1fbe5fb1312a7b2f8f25c03b3e07113782f7c12f00e023e148a\""
+      "Description": "S3 bucket for asset \"bbe209afddb09a12327bab7a105e085758a29b769b5b4bf5b6320ac41b05fc51\""
     },
-    "AssetParametersb13aad60258df1fbe5fb1312a7b2f8f25c03b3e07113782f7c12f00e023e148aS3VersionKey31422F11": {
+    "AssetParametersbbe209afddb09a12327bab7a105e085758a29b769b5b4bf5b6320ac41b05fc51S3VersionKey81BEC7FB": {
       "Type": "String",
-      "Description": "S3 key for asset version \"b13aad60258df1fbe5fb1312a7b2f8f25c03b3e07113782f7c12f00e023e148a\""
+      "Description": "S3 key for asset version \"bbe209afddb09a12327bab7a105e085758a29b769b5b4bf5b6320ac41b05fc51\""
     },
-    "AssetParametersb13aad60258df1fbe5fb1312a7b2f8f25c03b3e07113782f7c12f00e023e148aArtifactHash2897446E": {
+    "AssetParametersbbe209afddb09a12327bab7a105e085758a29b769b5b4bf5b6320ac41b05fc51ArtifactHashB77349F4": {
       "Type": "String",
-      "Description": "Artifact hash for asset \"b13aad60258df1fbe5fb1312a7b2f8f25c03b3e07113782f7c12f00e023e148a\""
+      "Description": "Artifact hash for asset \"bbe209afddb09a12327bab7a105e085758a29b769b5b4bf5b6320ac41b05fc51\""
     },
-    "AssetParameters28e3582cfc7c551f42435f44110c499fb2c415fe0f02a02a77139cf43edd1145S3BucketFDBB032A": {
+    "AssetParameters7bd7ccfde94a4ad94f13971f10733ee01020eb6d7538b7a5c2b05966db2c0aabS3Bucket2E4E0318": {
       "Type": "String",
-      "Description": "S3 bucket for asset \"28e3582cfc7c551f42435f44110c499fb2c415fe0f02a02a77139cf43edd1145\""
+      "Description": "S3 bucket for asset \"7bd7ccfde94a4ad94f13971f10733ee01020eb6d7538b7a5c2b05966db2c0aab\""
     },
-    "AssetParameters28e3582cfc7c551f42435f44110c499fb2c415fe0f02a02a77139cf43edd1145S3VersionKey58263016": {
+    "AssetParameters7bd7ccfde94a4ad94f13971f10733ee01020eb6d7538b7a5c2b05966db2c0aabS3VersionKeyF6AB1DF2": {
       "Type": "String",
-      "Description": "S3 key for asset version \"28e3582cfc7c551f42435f44110c499fb2c415fe0f02a02a77139cf43edd1145\""
+      "Description": "S3 key for asset version \"7bd7ccfde94a4ad94f13971f10733ee01020eb6d7538b7a5c2b05966db2c0aab\""
     },
-    "AssetParameters28e3582cfc7c551f42435f44110c499fb2c415fe0f02a02a77139cf43edd1145ArtifactHash592B4471": {
+    "AssetParameters7bd7ccfde94a4ad94f13971f10733ee01020eb6d7538b7a5c2b05966db2c0aabArtifactHash63670210": {
       "Type": "String",
-      "Description": "Artifact hash for asset \"28e3582cfc7c551f42435f44110c499fb2c415fe0f02a02a77139cf43edd1145\""
+      "Description": "Artifact hash for asset \"7bd7ccfde94a4ad94f13971f10733ee01020eb6d7538b7a5c2b05966db2c0aab\""
     }
   }
 }