Terraform updates nomad_job even if nothing changes.

[blmhemu]

Terraform Version

Terraform v1.5.2
on darwin_amd64
+ provider registry.terraform.io/hashicorp/nomad v2.0.0-beta.1

Nomad Version

Name         Address   Port  Status  Leader  Raft Version  Build  Datacenter  Region
{node}.global  {addr}  4648  alive   true    3             1.5.6  dc1         global

Provider Configuration

provider "nomad" {} # Configured by setting NOMAD_* env vars

Environment Variables

Do you have any Nomad specific environment variable set in the machine running Terraform?

NOMAD_ADDR=https://xx.xx.xx.xx:4646
NOMAD_CACERT=xxxxxx.pem
NOMAD_TOKEN=xxxxxxx

Affected Resource(s)

• nomad_job

Terraform Configuration Files

provider "nomad" {} # Configure by setting NOMAD_* env vars

# Sample test job
resource "nomad_job" "httpbin" {
  jobspec = file("${path.module}/jobs/httpbin.hcl")
}

A dead simple job:

job "httpbin" {
  region      = "global"
  datacenters = ["dc1"]
  type        = "service"
  group "httpbin" {
    count = 1

    network {
      port "http" {}
    }

    service {
      port     = "http"
      provider = "nomad"
      check {
        type     = "http"
        path     = "/"
        interval = "60s"
        timeout  = "5s"
      }
    }

    task "httpbin" {
      driver = "docker"
      config {
        image = "mccutchen/go-httpbin:v2.10.0"
        ports = ["http"]
      }
      env {
        PORT = "${NOMAD_PORT_http}"
      }
      resources {
        cpu    = 100
        memory = 32
      }
    }
  }
}

Debug Output

2023-07-15T16:48:49.160+0530 [INFO]  provider.terraform-provider-nomad_v2.0.0-beta.1_x5: 2023/07/15 16:48:49 [DEBUG] reading 
information for job "httpbin" in namespace "default": timestamp=2023-07-15T16:48:49.160+0530
2023-07-15T16:48:49.222+0530 [INFO]  provider.terraform-provider-nomad_v2.0.0-beta.1_x5: 2023/07/15 16:48:49 [DEBUG] found jo
b "httpbin" in namespace "default": timestamp=2023-07-15T16:48:49.222+0530
2023-07-15T16:48:49.222+0530 [INFO]  provider.terraform-provider-nomad_v2.0.0-beta.1_x5: 2023/07/15 16:48:49 [DEBUG] resource
JobCustomizeDiff: timestamp=2023-07-15T16:48:49.244+0530
2023-07-15T16:48:49.245+0530 [INFO]  provider.terraform-provider-nomad_v2.0.0-beta.1_x5: 2023/07/15 16:48:49 
2023-07-15T16:48:49.277+0530 [WARN]  Provider "registry.terraform.io/hashicorp/nomad" produced an invalid plan for nomad_job.
httpbin, but we are tolerating it because it is using the legacy plugin SDK.
    The following problems may be the cause of any confusing errors from downstream operations:
      - .deregister_on_destroy: planned value cty.True for a non-computed attribute
      - .deregister_on_id_change: planned value cty.True for a non-computed attribute
      - .hcl1: planned value cty.False for a non-computed attribute
      - .detach: planned value cty.True for a non-computed attribute

Panic Output

NA

Expected Behavior

No need to apply anything

Actual Behavior

  ~ update in-place

Terraform will perform the following actions:

  # nomad_job.httpbin will be updated in-place
  ~ resource "nomad_job" "httpbin" {
      ~ allocation_ids          = [
          - "3c2d8184-950c-c2ad-8d8b-552448f1656b",
        ] -> (known after apply)
        id                      = "httpbin"
      ~ modify_index            = "2621" -> (known after apply)
        name                    = "httpbin"
        # (10 unchanged attributes hidden)
    }

Steps to Reproduce

1. terraform apply
2. Re-format the jobspec (spaces, tabs).
3. terraform apply

Important Factoids

Single region, single DC, ACL enabled deployment. TLS enabled (not mTLS).

References

#290 is similar (or even same) but created this bug for a very small setup repro.

[blmhemu]

My $0.02 - Terraform job resource detaches (default behaviour) before completing the actual placement - hence, we may not get the alloc_id. But when read later, it may show alloc_id which is different from the prev run and tf tries to match it.

EDIT: I tried running terraform refresh and terraform plan still shows the change.
EDIT2: Could it be non-changing jobspec - e.x spaces, tabs, comments etc ?

[blmhemu]

Confirmed the root cause to be (inconsequential) changes in the jobspec

1. Created a new and fresh setup.
2. Did terraform apply using the above job spec.
3. Change spaces in the jobspec e.x: region = "global" to region = "global"
4. Did terraform apply
5. The above infinite change loop occurs.

Could it be a bug (missed edge case?) in jobspecDiffSuppress or resourceJobCustomizeDiff ?

[lgfa29]

Thanks for the extra info @blmhemu.

I'm fairly certain that has the same root cause as #290 and #356 should fix it.