[Bug]: compressionType is not a valid parameter in extra connection attributes

[shanmugahp]

Terraform Core Version

1.0.5

AWS Provider Version

4.8.0

Affected Resource(s)

aws_dms_endpoint

Expected Behavior

Terraform should not send the default value for compressionType and Resource is created

Actual Behavior

Endpoint fails to create as aws has removed the compressionType attribute for dms s3 as source endpoint . Terraform is sending default value of compressionType=None causing the call to fail.

      + cdc_path                         = "changedata"
      + compression_type                 = "NONE"
      + csv_delimiter                    = "|"
      + csv_row_delimiter                = "\\n"
      + data_format                      = "csv"
      + data_page_size                   = 1048576
      + date_partition_delimiter         = "slash"
      + date_partition_enabled           = false
      + date_partition_sequence          = "yyyymmdd"

Relevant Error/Panic Output Snippet

Error: Error creating DMS endpoint: InvalidParameterValueException: Unsupported value 'compressionType' for extra connection attributes
	status code: 400, request id: 89916c08-3fa2-454e-affb-2c69c6572df1

  with module.data-lake.module.sources.module.source_activities[0].aws_dms_endpoint.main[0],
  on ../../main/data-sources/s3-data-source/main.tf line 5, in resource "aws_dms_endpoint" "main":
   5: resource "aws_dms_endpoint" "main" {

Terraform Configuration Files

resource "aws_dms_endpoint" "main" {

endpoint_id = var.resource_prefix
endpoint_type = "source"
ssl_mode = "none"
tags = var.application_tags
engine_name = "s3"

s3_settings {
bucket_name = var.source_bucket_id
bucket_folder = var.source_bucket_folder
service_access_role_arn = var.activities_dms_s3_role_arn
external_table_definition = var.external_table_mappings
csv_delimiter = var.csv_delimiter
cdc_path = var.cdc_path
csv_null_value = var.csv_null_value
csv_row_delimiter = var.csv_row_delimiter
}

Steps to Reproduce

Create a DMS source endpoint with S3 as source

Debug Output

No response

Panic Output

No response

Important Factoids

No response

References

• DMS Endpoint Re-design #23506

Would you like to implement a fix?

No response

[github-actions]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[JoeHutchins0n]

Same problem here. The Terraform resource for the DMS S3 endpoint I was testing broke overnight. Very frustrating.

I compiled a local copy of the provider and found at least 3 other attributes that have had their support removed as well as CompressionType:

• DatePartitionDelimiter
• DatePartitionEnabled
• DatePartitionSequence

There could be more to.

[rossbush]

Our pipelines are silent. Please make this a high priority.

[jeffarredondo]

Ok at least I'm not crazy. Ran into something similar for MySQL as well when testing a potential change to include the sybase engine

[thebrettd]

I believe I am being impacted by this also.

I believe there may be a related issue with an old fix: #14620

[YakDriver]

AWS has been warning for some time in the console that extra connection attributes would be deprecated (see #23506). Unfortunately, work on new endpoint resources has stalled (#23507). Thank you for bringing these challenges to our attention as we prioritize work.

[github-actions]

This functionality has been released in v4.45.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.