Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

aws_dynamodb_table - with ttl disabled, can't "terraform apply" twice #10304

Closed
ghost opened this issue Sep 30, 2019 · 31 comments · Fixed by #37991
Closed

aws_dynamodb_table - with ttl disabled, can't "terraform apply" twice #10304

ghost opened this issue Sep 30, 2019 · 31 comments · Fixed by #37991
Assignees
Labels
bug Addresses a defect in current functionality. service/dynamodb Issues and PRs that pertain to the dynamodb service.
Milestone

Comments

@ghost
Copy link

ghost commented Sep 30, 2019

This issue was originally opened by @nunoperalta as hashicorp/terraform#22942. It was migrated here as a result of the provider split. The original body of the issue is below.


Terraform Version

Terraform v0.12.9
+ provider.aws v2.30.0

Terraform Configuration Files

resource "aws_dynamodb_table" "dynamodb-testdb" {
  name           = "TestDb"
  billing_mode   = "PAY_PER_REQUEST"
  hash_key       = "PriKey"
  range_key      = "Quantity"

  attribute {
    name = "PriKey"
    type = "S"
  }

  attribute {
    name = "Quantity"
    type = "N"
  }

  ttl {
    attribute_name = "TimeToExist"
    enabled        = false
  }
}

Expected Behavior

When doing "terraform apply" twice, there should be no changes to make.

Actual Behavior

First "terraform apply" will create the DynamoDB table.
However, second time, there will be a change at:

      ~ ttl {
          + attribute_name = "TimeToExist"
            enabled        = false
        }

If I confirm the change, I get this crash:

Error: error updating DynamoDB Table (TestDb) time to live: error updating DynamoDB Table (TestDb) Time To Live: ValidationException: TimeToLive is already disabled
status code: 400, request id: XXXXXX

If I remove the "attribute_name", I get this error:

The argument "attribute_name" is required, but no definition was found.

@ghost ghost added the service/dynamodb Issues and PRs that pertain to the dynamodb service. label Sep 30, 2019
@github-actions github-actions bot added the needs-triage Waiting for first response or review from a maintainer. label Sep 30, 2019
@inge4pres
Copy link

inge4pres commented Nov 22, 2019

Just to give more context here and to #3463

I just found out talking with AWS that the DynamoDB TTL API is batched, meaning that there is no way without to set atomically AttributeName and Enabled. Running the same operation through AWS CLI will result in the same 400 BadRequest errors because that's just how Dynamo DB TTL API works.

It is normal for DynamoDB API to reply with an error when trying to disable an already-disabeld TTL; and it's normal to not be able to enable/disable in a short period of time, you would get this error form CLI

➜  ~ aws dynamodb update-time-to-live --table-name my-table --time-to-live-specification "Enabled=true, AttributeName=UpdateTime"

An error occurred (ValidationException) when calling the UpdateTimeToLive operation: Time to live has been modified multiple times within a fixed interval

This is a PITA for Terraform because it means that when operating through the DynamoDB TTL settings we should have a polling or a time-bound control that we can actually perform an atomic operation.
I'll chat again with AWS soon and hopefully prepare a pull request with a proposed fix...

@inge4pres
Copy link

As per this doc page

It can take up to one hour for the change to fully process. Any additional UpdateTimeToLive calls for the same table during this one hour duration result in a ValidationException. 

so we can't really have this state reconciliation loop in Terraform...

@MattMcKeithen
Copy link

MattMcKeithen commented Apr 14, 2020

The "ValidationException: TimeToLive is already disabled" error is quite annoying if you are trying to modularize this.
As a work around, consider using a dynamic block.

credit: https://www.reddit.com/r/Terraform/comments/d1va2o/terrfaorm_support_null_block/

locals {
  ttl = (var.ttl_enable == true ? [
    {
      ttl_enable = var.ttl_enable
      ttl_attribute : var.ttl_attribute
    }
  ] : [])
}

...

  dynamic "ttl" {
    for_each = local.ttl
    content {
      enabled        = local.ttl[0].ttl_enable
      attribute_name = local.ttl[0].ttl_attribute
    }
  }

@ORDEP81
Copy link

ORDEP81 commented Jul 11, 2021

Curious, was there an actual fix to this?

@justinretzolk
Copy link
Member

Hey y'all 👋 Thank you for taking the time to file this issue and for the continued discussion! Given that there's been a number of AWS provider releases since it was initially filed, can anyone confirm whether you're still experiencing this behavior?

@justinretzolk justinretzolk added waiting-response Maintainers are waiting on response from community or contributor. and removed needs-triage Waiting for first response or review from a maintainer. labels Dec 9, 2021
@poolieweb
Copy link

Still happening.

@github-actions github-actions bot removed the waiting-response Maintainers are waiting on response from community or contributor. label Dec 13, 2021
@bgshacklett
Copy link

Perhaps the documentation should be updated so that this isn't the default example.

https://github.com/hashicorp/terraform-provider-aws/blob/main/website/docs/r/dynamodb_table.html.markdown?plain=1#L44-L47

@justinretzolk justinretzolk added the bug Addresses a defect in current functionality. label Dec 16, 2021
@oldskyba
Copy link

The issue is still happening.
I looked at the tfstate and the attribute name is not set if it is disabled.

@DevSusu
Copy link

DevSusu commented Apr 5, 2022

Still happening here ✋

@jakekdodd
Copy link

Still happening with provider version 4.10.0

@vineetsharma883
Copy link

still happening in 4.9.0

@dany74q
Copy link

dany74q commented Jul 12, 2022

^ +1

@sumeetninawe
Copy link

Still happening..

@jonnymccullagh
Copy link

still happening

2 similar comments
@drewmobile
Copy link

still happening

@grantjoy
Copy link

grantjoy commented Nov 2, 2022

still happening

@amirkav
Copy link

amirkav commented Dec 12, 2022

Still happening

1 similar comment
@mmontero-twilio
Copy link

Still happening

@SupornoChaudhury
Copy link

SupornoChaudhury commented Jan 4, 2023

I am still getting this error. And the only workaround seems to be the one suggested by @MattMcKeithen of using dynamic block. (not tried it yet)
Does anyone use any other workaround? It is of utmost importance for us to keep the TTL disabled.

@CyrilDevOps
Copy link

CyrilDevOps commented Jan 5, 2023

Sadly a dynamic block doesn't work,
with a block like

dynamic "ttl" {
    for_each = var.ttl_enabled ? [1] : []
    content {
      attribute_name = var.ttl_attribute
      enabled        = var.ttl_enabled
    }
  }

If you enable ttl it work, but then if you disable it, the block disappear and terraform doesn't see you want a enabled=false and it doesn't report any change and AWS keep the ttl settings.

I see that as a bug.

  • allow enable = false and no attribute_name, and/or
  • enforce enable = false when the ttl block isn't present.

@sathishdv
Copy link

Still happening

1 similar comment
@shasi24
Copy link

shasi24 commented Mar 7, 2023

Still happening

@alfredo-gil
Copy link

alfredo-gil commented Apr 14, 2023

Still happening

@dth138
Copy link

dth138 commented May 19, 2023

still happening

@kennethjmyers
Copy link

kennethjmyers commented Aug 10, 2023

This isn't really a "fix" but here is what I did to get it to not error (you probably only need step 3 but this is how I got there):

  1. Removed the dynamo definitions from tfstate
  2. Run terraform import for the tables I had just removed from tfstate. In the newly imported definitions the definition looked as follows:
    "ttl": [
              {
                "attribute_name": "",
                "enabled": false
              }
            ]
    
    This is different because the example I copied from documentation was as follows:
    ttl {
      attribute_name = "TimeToExist"
      enabled        = false
    }
    
  3. So in my new definition I just converted "TimeToExist" to an empty string "" and when I did terraform plan it no longer wanted to make changes.

Sorry if someone already said this above but seems like the issue comes from "disabled" ttl not needing or using the supplied attribute name, but the resource definition requires an attribute name and many people probably just copy the example which gives one. So when you try to apply an update, it sees a "new" attribute definition it didn't use the first time. The documentation example should really be fixed to reflect this and the documentation should explain this quirk.

@rg-lesmills
Copy link

@kennethjmyers I have tried setting attribute_name to an empty string as suggested, but the apply gets rejected by AWS with a validation error:

Error: updating Amazon DynamoDB Table (ronnies-component-RonnieDB-sandbox): updating Time To Live: InvalidParameter: 1 validation error(s) found.
│ - minimum field size of 1, UpdateTimeToLiveInput.TimeToLiveSpecification.AttributeName.

Looking at the plan, it seems like terraform is taking a blank string in the table config and turning that into null:

      ~ ttl {
          - attribute_name = "ttl_field" -> null
          ~ enabled        = true -> false
        }

I'm using the typescript CDK @cdktf/provider-aws module 16.03 which is provider version 5.8.0

Steps to reproduce:

  1. create a table with ttl.enabled = true and ttl.attribute_name whatever
  2. set ttl.enabled = false and tt.attribute_name = ''
  3. terraform apply results in minimum field size of 1 validation error as above

In order to actually disable the TTL and have terraform stop trying to re-add attribute_name as in the OP of this issue, I need to follow these steps:

  1. create a table with ttl.enabled = true and ttl.attribute_name whatever
  2. set ttl.enabled = false and leave ttl.attribute_name unchanged
  3. terraform apply results in correct update to DynamoDB table
  4. now set ttl.attribute_name to empty string
  5. subsequent terraform plan/apply shows no changes required, as desired

@vgw-chriskruger
Copy link

vgw-chriskruger commented Oct 24, 2023

This problem has existed for years on this thread alone and this is the new issue for this problem. This is just one of dozens of open bugs annoying my team with terraform at this juncture.

@ek2020
Copy link

ek2020 commented Dec 26, 2023

Still happening, Curious, was there an actual fix to this? opened - Sep 30, 2019 - Now( Dec 26, 2023 ).

@pscosta
Copy link

pscosta commented May 29, 2024

Still happening , sadly.

@gdavison gdavison self-assigned this Jun 14, 2024
@terraform-aws-provider terraform-aws-provider bot added the prioritized Part of the maintainer teams immediate focus. To be addressed within the current quarter. label Jun 14, 2024
Copy link

Warning

This issue has been closed, meaning that any additional comments are hard for our team to see. Please assume that the maintainers will not see them.

Ongoing conversations amongst community members are welcome, however, the issue will be locked after 30 days. Moving conversations to another venue, such as the AWS Provider forum, is recommended. If you have additional concerns, please open a new issue, referencing this one where needed.

@github-actions github-actions bot added this to the v5.55.0 milestone Jun 17, 2024
@github-actions github-actions bot removed the prioritized Part of the maintainer teams immediate focus. To be addressed within the current quarter. label Jun 20, 2024
Copy link

This functionality has been released in v5.55.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Addresses a defect in current functionality. service/dynamodb Issues and PRs that pertain to the dynamodb service.
Projects
None yet