Update typespec to support all Terraform Recipe Providers #7199

ytimocin · 2024-02-25T20:56:40Z

Description

Did some refactoring
Updated TypeSpec by adding Providers and EnvVariables to Environment resource
Updated Data Models by adding Providers and EnvVariables to Environment resource
Updated the Environment Conversion logic and its tests
Updated RecipeConfigProperties

Design doc: radius-project/design-notes#39

Type of change

This pull request adds or changes features of Radius and has an approved issue (issue link required).
Fixes: part of Allow any Terraform provider to be configured and used in a Recipe #6539

radius-functional-tests · 2024-02-25T21:34:22Z

Radius functional test overview

🔍 Go to test action run

Name	Value
Repository	vishwahiremat/radius
Commit ref	`e8a8bc2`
Unique ID	ca3c92c713
Image tag	pr-ca3c92c713

Click here to see the list of tools in the current test run

gotestsum 1.10.0
KinD: v0.20.0
Dapr: 1.12.0
Azure KeyVault CSI driver: 1.4.2
Azure Workload identity webhook: 1.1.0
Bicep recipe location ghcr.io/radius-project/dev/test/functional/shared/recipes/<name>:pr-ca3c92c713
Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-ca3c92c713
controller test image location: ghcr.io/radius-project/dev/controller:pr-ca3c92c713
ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-ca3c92c713
deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting datastoresrp functional tests...
⌛ Starting ucp functional tests...
⌛ Starting msgrp functional tests...
⌛ Starting daprrp functional tests...
⌛ Starting shared functional tests...
⌛ Starting samples functional tests...
⌛ Starting kubernetes functional tests...
✅ msgrp functional tests succeeded
✅ samples functional tests succeeded
✅ kubernetes functional tests succeeded
✅ datastoresrp functional tests succeeded
✅ ucp functional tests succeeded
✅ daprrp functional tests succeeded
✅ shared functional tests succeeded

kachawla

Could you look into comments on the original version of this PR vishwahiremat#1?

kachawla

Please update the PR title and description to provide context on the change. It is in general good practice to write the description in a way that anyone can understand what the PR is about, think about someone looking at the commit in the future to understand what this change is, why it was etc.

...rp/api/v20231001preview/testdata/environmentrecipepropertiesdatamodel-insecure-registry.json

radius-functional-tests · 2024-02-26T20:36:15Z

Radius functional test overview

🔍 Go to test action run

Name	Value
Repository	vishwahiremat/radius
Commit ref	`6ff48cd`
Unique ID	55e5f1b88e
Image tag	pr-55e5f1b88e

Click here to see the list of tools in the current test run

gotestsum 1.10.0
KinD: v0.20.0
Dapr: 1.12.0
Azure KeyVault CSI driver: 1.4.2
Azure Workload identity webhook: 1.1.0
Bicep recipe location ghcr.io/radius-project/dev/test/functional/shared/recipes/<name>:pr-55e5f1b88e
Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-55e5f1b88e
controller test image location: ghcr.io/radius-project/dev/controller:pr-55e5f1b88e
ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-55e5f1b88e
deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting daprrp functional tests...
⌛ Starting samples functional tests...
⌛ Starting shared functional tests...
⌛ Starting msgrp functional tests...
⌛ Starting kubernetes functional tests...
⌛ Starting ucp functional tests...
⌛ Starting datastoresrp functional tests...
✅ samples functional tests succeeded
✅ kubernetes functional tests succeeded
✅ msgrp functional tests succeeded
✅ daprrp functional tests succeeded
✅ ucp functional tests succeeded
✅ datastoresrp functional tests succeeded
✅ shared functional tests succeeded

kachawla · 2024-02-26T21:33:33Z

Please update the PR title and description to provide context on the change. It is in general good practice to write the description in a way that anyone can understand what the PR is about, think about someone looking at the commit in the future to understand what this change is, why it was etc.

Saw the updated title - The title should include a brief description of the functionality this PR is adding and the component it is touching. Details of which files/components are updated should be in the description. I have updated the title from Update typespec, datamodel, converter, and tests for Provider and EnvironmentVariables to Update typespec to support all Terraform Recipe Providers. And have included link to the design doc in the description.

When writing the description, I'd suggest think about someone who is fairly new to the repo, can they tell the summary of why and what changes are made by looking at the commit description?

typespec/Applications.Core/environments.tsp

pkg/recipes/terraform/execute.go

radius-functional-tests · 2024-02-26T23:57:53Z

Radius functional test overview

🔍 Go to test action run

Name	Value
Repository	vishwahiremat/radius
Commit ref	`3163303`
Unique ID	1632767d41
Image tag	pr-1632767d41

Click here to see the list of tools in the current test run

gotestsum 1.10.0
KinD: v0.20.0
Dapr: 1.12.0
Azure KeyVault CSI driver: 1.4.2
Azure Workload identity webhook: 1.1.0
Bicep recipe location ghcr.io/radius-project/dev/test/functional/shared/recipes/<name>:pr-1632767d41
Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-1632767d41
controller test image location: ghcr.io/radius-project/dev/controller:pr-1632767d41
ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-1632767d41
deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting shared functional tests...
⌛ Starting samples functional tests...
⌛ Starting ucp functional tests...
⌛ Starting msgrp functional tests...
⌛ Starting kubernetes functional tests...
⌛ Starting daprrp functional tests...
✅ msgrp functional tests succeeded
✅ samples functional tests succeeded
✅ kubernetes functional tests succeeded
✅ ucp functional tests succeeded
✅ datastoresrp functional tests succeeded
✅ daprrp functional tests succeeded
✅ shared functional tests succeeded

typespec/Applications.Core/environments.tsp

kachawla · 2024-02-27T01:58:23Z

pkg/corerp/api/v20231001preview/testdata/environmentresourcedatamodel.json

              }
            }
          }
+        },
+        "providers": {


do we have test coverage for empty values for these?

we have empty data model and resource examples.

pkg/corerp/datamodel/recipe_types.go

radius-functional-tests · 2024-02-27T03:57:29Z

Radius functional test overview

🔍 Go to test action run

Name	Value
Repository	vishwahiremat/radius
Commit ref	`6ee7274`
Unique ID	ff7cf5e174
Image tag	pr-ff7cf5e174

Click here to see the list of tools in the current test run

gotestsum 1.10.0
KinD: v0.20.0
Dapr: 1.12.0
Azure KeyVault CSI driver: 1.4.2
Azure Workload identity webhook: 1.1.0
Bicep recipe location ghcr.io/radius-project/dev/test/functional/shared/recipes/<name>:pr-ff7cf5e174
Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-ff7cf5e174
controller test image location: ghcr.io/radius-project/dev/controller:pr-ff7cf5e174
ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-ff7cf5e174
deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting daprrp functional tests...
⌛ Starting samples functional tests...
⌛ Starting kubernetes functional tests...
⌛ Starting msgrp functional tests...
⌛ Starting datastoresrp functional tests...
⌛ Starting ucp functional tests...
⌛ Starting shared functional tests...
✅ msgrp functional tests succeeded
✅ samples functional tests succeeded
✅ kubernetes functional tests succeeded
✅ ucp functional tests succeeded
✅ daprrp functional tests succeeded
✅ datastoresrp functional tests succeeded
✅ shared functional tests succeeded

radius-functional-tests · 2024-02-27T07:31:47Z

Radius functional test overview

🔍 Go to test action run

Name	Value
Repository	vishwahiremat/radius
Commit ref	`7b3eb9e`
Unique ID	bc131d25e6
Image tag	pr-bc131d25e6

Click here to see the list of tools in the current test run

gotestsum 1.10.0
KinD: v0.20.0
Dapr: 1.12.0
Azure KeyVault CSI driver: 1.4.2
Azure Workload identity webhook: 1.1.0
Bicep recipe location ghcr.io/radius-project/dev/test/functional/shared/recipes/<name>:pr-bc131d25e6
Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-bc131d25e6
controller test image location: ghcr.io/radius-project/dev/controller:pr-bc131d25e6
ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-bc131d25e6
deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting shared functional tests...
⌛ Starting daprrp functional tests...
⌛ Starting datastoresrp functional tests...
⌛ Starting kubernetes functional tests...
⌛ Starting samples functional tests...
⌛ Starting msgrp functional tests...
✅ samples functional tests succeeded
✅ kubernetes functional tests succeeded
✅ ucp functional tests succeeded
✅ msgrp functional tests succeeded
✅ daprrp functional tests succeeded
✅ datastoresrp functional tests succeeded
✅ shared functional tests succeeded

rynowak · 2024-02-27T17:22:50Z

typespec/Applications.Core/environments.tsp

@@ -77,23 +78,29 @@ model EnvironmentProperties {
 model RecipeConfigProperties {
  @doc("Configuration for Terraform Recipes. Controls how Terraform plans and applies templates as part of Recipe deployment.")
  terraform?: TerraformConfigProperties;
+
+  @doc("Environment variables injected during Terraform Recipe execution for the recipes in the environment.")
+  envVariables?: EnvironmentVariables;


This should be env like the design doc. https://github.com/radius-project/design-notes/pull/39/files#diff-f9f52b8c3872a6c388dff224c472dfdd39bb011c139480ed60e7f4facd68aa10R183

We review API changes in detail because they are hard to change later. Please do a pass on on this and make sure that the API changes match the design doc.

I think this drifted from design doc after vishwahiremat#1 (comment) @lakshmimsft thoughts?

If there is drift from an approved design doc, then it needs to come back to the design meeting. We review details like naming for clarity and consistency.

If there is drift from an approved design doc, then it needs to come back to the design meeting. We review details like naming for clarity and consistency.

Completely agree. This was a miss.

rynowak · 2024-02-27T17:24:45Z

typespec/Applications.Core/environments.tsp

+}
+
+@doc("This secret is used within a recipe. Secrets are encrypted, often have fine-grained access control, auditing and are recommended to be used to hold sensitive data.")
+model RecipeSecret {


The design doc calls this ProviderSecret.

This was me here vishwahiremat#1 (comment). @rynowak if you think ProviderSecret makes more sense and we should add new properties for secrets for anything else that we might introduce in the future, then keeping it as ProviderSecret sounds good to me.

I think should have brought this up for a re-review of api spec design if it absolutely needed to change.

@lakshmimsft @ytimocin Let's re-review this and env in the design review today.

youngbupark · 2024-02-27T18:39:12Z

@ytimocin When you correct indentation on the multiple files, please file the separate PR.

Signed-off-by: ytimocin <ytimocin@microsoft.com>

radius-functional-tests · 2024-02-27T18:42:38Z

Radius functional test overview

🔍 Go to test action run

Name	Value
Repository	vishwahiremat/radius
Commit ref	`1188d65`
Unique ID	4512151054
Image tag	pr-4512151054

Click here to see the list of tools in the current test run

gotestsum 1.10.0
KinD: v0.20.0
Dapr: 1.12.0
Azure KeyVault CSI driver: 1.4.2
Azure Workload identity webhook: 1.1.0
Bicep recipe location ghcr.io/radius-project/dev/test/functional/shared/recipes/<name>:pr-4512151054
Terraform recipe location http://tf-module-server.radius-test-tf-module-server.svc.cluster.local/<name>.zip (in cluster)
applications-rp test image location: ghcr.io/radius-project/dev/applications-rp:pr-4512151054
controller test image location: ghcr.io/radius-project/dev/controller:pr-4512151054
ucp test image location: ghcr.io/radius-project/dev/ucpd:pr-4512151054
deployment-engine test image location: ghcr.io/radius-project/deployment-engine:latest

Test Status

⌛ Building Radius and pushing container images for functional tests...
✅ Container images build succeeded
⌛ Publishing Bicep Recipes for functional tests...
✅ Recipe publishing succeeded
⌛ Starting datastoresrp functional tests...
⌛ Starting kubernetes functional tests...
⌛ Starting ucp functional tests...
⌛ Starting msgrp functional tests...
⌛ Starting shared functional tests...
⌛ Starting samples functional tests...
⌛ Starting daprrp functional tests...
✅ msgrp functional tests succeeded
✅ kubernetes functional tests succeeded
✅ ucp functional tests succeeded
✅ samples functional tests succeeded
✅ datastoresrp functional tests succeeded
✅ daprrp functional tests succeeded
✅ shared functional tests succeeded

ytimocin · 2024-02-28T01:00:08Z

PR will be closed but branch SHOULDN'T BE DELETED.

ytimocin requested review from a team as code owners February 25, 2024 20:56

ytimocin force-pushed the ytimocin/multipleTFProviders2 branch from 27ce24d to e8a8bc2 Compare February 25, 2024 21:32

ytimocin temporarily deployed to functional-tests February 25, 2024 21:33 — with GitHub Actions Inactive

kachawla reviewed Feb 26, 2024

View reviewed changes

ytimocin force-pushed the ytimocin/multipleTFProviders2 branch 2 times, most recently from b55ecf4 to fbdba5c Compare February 26, 2024 16:21

kachawla reviewed Feb 26, 2024

View reviewed changes

ytimocin force-pushed the ytimocin/multipleTFProviders2 branch 2 times, most recently from 8dab28e to 6ff48cd Compare February 26, 2024 20:33

ytimocin commented Feb 26, 2024

View reviewed changes

...rp/api/v20231001preview/testdata/environmentrecipepropertiesdatamodel-insecure-registry.json Outdated Show resolved Hide resolved

ytimocin temporarily deployed to functional-tests February 26, 2024 20:35 — with GitHub Actions Inactive

ytimocin changed the title ~~Initial work on adding support for the multiple TF Providers~~ Update typespec, datamodel, converter, and tests for Provider and EnvironmentVariables Feb 26, 2024

kachawla changed the title ~~Update typespec, datamodel, converter, and tests for Provider and EnvironmentVariables~~ Update typespec to support all Terraform Recipe Providers Feb 26, 2024

kachawla reviewed Feb 26, 2024

View reviewed changes

typespec/Applications.Core/environments.tsp Outdated Show resolved Hide resolved

kachawla reviewed Feb 26, 2024

View reviewed changes

lakshmimsft reviewed Feb 26, 2024

View reviewed changes

pkg/recipes/terraform/execute.go Outdated Show resolved Hide resolved

ytimocin force-pushed the ytimocin/multipleTFProviders2 branch 3 times, most recently from aa5bfd8 to 3163303 Compare February 26, 2024 23:56

ytimocin temporarily deployed to functional-tests February 26, 2024 23:57 — with GitHub Actions Inactive

ytimocin requested review from kachawla, vishwahiremat and lakshmimsft February 27, 2024 00:04

kachawla reviewed Feb 27, 2024

View reviewed changes

typespec/Applications.Core/environments.tsp Outdated Show resolved Hide resolved

typespec/Applications.Core/environments.tsp Show resolved Hide resolved

typespec/Applications.Core/environments.tsp Show resolved Hide resolved

kachawla reviewed Feb 27, 2024

View reviewed changes

ytimocin force-pushed the ytimocin/multipleTFProviders2 branch from 3163303 to 6ee7274 Compare February 27, 2024 03:54

ytimocin temporarily deployed to functional-tests February 27, 2024 03:56 — with GitHub Actions Inactive

ytimocin requested a review from kachawla February 27, 2024 06:27

ytimocin force-pushed the ytimocin/multipleTFProviders2 branch from 6ee7274 to 7b3eb9e Compare February 27, 2024 07:30

ytimocin temporarily deployed to functional-tests February 27, 2024 07:31 — with GitHub Actions Inactive

rynowak reviewed Feb 27, 2024

View reviewed changes

Initial work on adding support for the multiple TF Providers

1188d65

Signed-off-by: ytimocin <ytimocin@microsoft.com>

ytimocin force-pushed the ytimocin/multipleTFProviders2 branch from 7b3eb9e to 1188d65 Compare February 27, 2024 18:40

ytimocin temporarily deployed to functional-tests February 27, 2024 18:41 — with GitHub Actions Inactive

kachawla mentioned this pull request Feb 28, 2024

Update typespec to support all Terraform Recipe Providers and Env #7202

Merged

ytimocin closed this Feb 28, 2024

ytimocin reopened this Jun 25, 2024

ytimocin closed this Jul 8, 2024

kachawla mentioned this pull request Jul 17, 2024

Add datamodel changes for secret support to Terraform Providers #7731

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update typespec to support all Terraform Recipe Providers #7199

Update typespec to support all Terraform Recipe Providers #7199

ytimocin commented Feb 25, 2024 •

edited by kachawla

Loading

radius-functional-tests bot commented Feb 25, 2024 •

edited

Loading

kachawla left a comment

kachawla left a comment

radius-functional-tests bot commented Feb 26, 2024 •

edited

Loading

kachawla commented Feb 26, 2024 •

edited

Loading

radius-functional-tests bot commented Feb 26, 2024 •

edited

Loading

kachawla Feb 27, 2024

ytimocin Feb 27, 2024

radius-functional-tests bot commented Feb 27, 2024 •

edited

Loading

radius-functional-tests bot commented Feb 27, 2024 •

edited

Loading

rynowak Feb 27, 2024

kachawla Feb 27, 2024

rynowak Feb 27, 2024

kachawla Feb 27, 2024

rynowak Feb 27, 2024

kachawla Feb 27, 2024 •

edited

Loading

kachawla Feb 27, 2024

youngbupark commented Feb 27, 2024

radius-functional-tests bot commented Feb 27, 2024 •

edited

Loading

ytimocin commented Feb 28, 2024

Update typespec to support all Terraform Recipe Providers #7199

Update typespec to support all Terraform Recipe Providers #7199

Conversation

ytimocin commented Feb 25, 2024 • edited by kachawla Loading

Description

Type of change

radius-functional-tests bot commented Feb 25, 2024 • edited Loading

Radius functional test overview

Test Status

kachawla left a comment

Choose a reason for hiding this comment

kachawla left a comment

Choose a reason for hiding this comment

radius-functional-tests bot commented Feb 26, 2024 • edited Loading

Radius functional test overview

Test Status

kachawla commented Feb 26, 2024 • edited Loading

radius-functional-tests bot commented Feb 26, 2024 • edited Loading

Radius functional test overview

Test Status

Choose a reason for hiding this comment

Choose a reason for hiding this comment

radius-functional-tests bot commented Feb 27, 2024 • edited Loading

Radius functional test overview

Test Status

radius-functional-tests bot commented Feb 27, 2024 • edited Loading

Radius functional test overview

Test Status

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kachawla Feb 27, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

youngbupark commented Feb 27, 2024

radius-functional-tests bot commented Feb 27, 2024 • edited Loading

Radius functional test overview

Test Status

ytimocin commented Feb 28, 2024

ytimocin commented Feb 25, 2024 •

edited by kachawla

Loading

radius-functional-tests bot commented Feb 25, 2024 •

edited

Loading

radius-functional-tests bot commented Feb 26, 2024 •

edited

Loading

kachawla commented Feb 26, 2024 •

edited

Loading

radius-functional-tests bot commented Feb 26, 2024 •

edited

Loading

radius-functional-tests bot commented Feb 27, 2024 •

edited

Loading

radius-functional-tests bot commented Feb 27, 2024 •

edited

Loading

kachawla Feb 27, 2024 •

edited

Loading

radius-functional-tests bot commented Feb 27, 2024 •

edited

Loading