Update validate bicep workflow #7702

sk593 · 2024-06-20T20:40:57Z

Description

Update validate bicep workflow

Type of change

This pull request fixes a bug in Radius and has an approved issue (issue link required).
This pull request adds or changes features of Radius and has an approved issue (issue link required).
This pull request is a minor refactor, code cleanup, test improvement, or other maintenance task and doesn't change the functionality of Radius (issue link optional).

Fixes: #issue_number

sk593 · 2024-06-20T20:41:23Z

build/validate-bicep.sh

@@ -5,6 +5,7 @@ then
    exit 1
 fi

+WARNING_MSG="WARNING: The following experimental Bicep features have been enabled: Extensibility, Dynamic type loading, Provider registry. Experimental features should be enabled for testing purposes only, as there are no guarantees about the quality or stability of these features. Do not enable these settings for any production usage, or your production environment may be subject to breaking."


Will look into if this can be disabled so it doesn't print on every build and we're not hardcoding the value

For now if this is the output, we'll ignore it since it's not an error

What if this message changes? How will we know that we need to update this?

The workflow would start failing. I think the ideal situation would be to suppress the warning message if that's possible

Possible we could turn this into a wildcard so we don't have to match the whole string? eg: WARNING: The following experimental Bicep features *

I'm wondering about whether the string is going to change and break our build.

sk593 · 2024-06-20T21:27:32Z

This was a casing issue in the generator. Fixed this as part of the PR

The Bicep compiler doesn't make assumptions about the location (rad CLI defaults to 'global') so we'll include that in our bicep templates

ytimocin · 2024-06-21T19:49:48Z

build/validate-bicep.sh

@@ -5,6 +5,7 @@ then
    exit 1
 fi

+WARNING_MSG="WARNING: The following experimental Bicep features have been enabled: Extensibility, Dynamic type loading, Provider registry. Experimental features should be enabled for testing purposes only, as there are no guarantees about the quality or stability of these features. Do not enable these settings for any production usage, or your production environment may be subject to breaking."


What if this message changes? How will we know that we need to update this?

ytimocin · 2024-06-21T19:51:56Z

...p/noncloud/resources/testdata/containers/corerp-resources-friendly-container-version-1.bicep

@@ -19,7 +19,7 @@ resource webapp 'Applications.Core/containers@2023-10-01-preview' = {
    container: {
      image: magpieimage
      env: {
-        DBCONNECTION: redis.connectionString()
+        DBCONNECTION: redis.listSecrets().connectionString


Was this wrong before?

Or is it a new feature of the provider?

Yes, its a new feature so we now make function calls using listSecrets

Yes, we're intentionally simplifying the design of the provider.

ytimocin · 2024-06-21T19:55:37Z

test/testrecipes/test-bicep-recipes/resource-creation.bicep

@@ -8,6 +8,7 @@ var basename = context.resource.name
 // doing things this way to test the UCP functionality without using cloud resources.
 resource extender 'Applications.Core/extenders@2023-10-01-preview' = {
  name: '${basename}-created'
+  location: 'global'


Should this be a constant?

rynowak · 2024-06-24T16:20:06Z

bicepconfig.json

+		"dynamicTypeLoading": true
+	},
+	"providers": {
+		"radius": "br:biceptypes.azurecr.io/radius:latest",


How do you expect this to work for a PR?

The bicepconfig here is the default for the repo and it shouldn't change (so when you're doing development work, the templates will all compile). These will all use the latest tag. For PRs, we'll have a temporary bicepconfig that's generated with a PR-specific tag for workflows like functional testing, PR builds, etc. That'll be added in a separate PR

rynowak · 2024-06-24T16:25:34Z

...ional-portable/corerp/cloud/resources/testdata/corerp-resources-extender-aws-s3-recipe.bicep


 resource env 'Applications.Core/environments@2023-10-01-preview' = {
  name: 'corerp-resources-extenders-aws-s3-recipe-env'
-  location: 'global'
+  location: location


Is there a reason why we need to specify the location here? We made a change to make location optional in Bicep for our resources.

If location is not showing as optional, then that's a regression and we need to fix it.

Location has as required flag in the types files so this might be a regression in the generation logic. Will look into that and fix it. But our docs also have location as a required field so that probably needs to get updated too: https://docs.radapp.io/reference/resource-schema/overview/

Whether or not it's required depends on which client you're talking about. The Radius API requires location to be set. Bicep will default it for you. One of the ways we tweak the Bicep experience to be simpler.

This was a casing bug in the generator. Fixed as part of the PR

github-actions · 2024-06-26T23:10:35Z

Unit Tests

3 219 tests ±0 3 213 ✅ ±0 3m 50s ⏱️ -7s
261 suites ±0 6 💤 ±0
1 files ±0 0 ❌ ±0

Results for commit 64bb10c. ± Comparison against base commit 0cc2c64.

♻️ This comment has been updated with latest results.

Need to take a look one more time. Approved by mistake. Looking at it now.

Signed-off-by: sk593 <shruthikumar@microsoft.com>

willdavsmith

approved with minor questions

willdavsmith · 2024-07-11T16:21:49Z

.github/workflows/build.yaml

@@ -283,7 +283,7 @@ jobs:
          node-version: '18'
      - name: Generate Bicep extensibility types from OpenAPI specs
        run: |
-          make generate-bicep-types VERSION=${{ env.REL_VERSION == 'edge' && 'latest' || env.REL_VERSION }}
+          make generate-bicep-types VERSION=${{ env.REL_CHANNEL == 'edge' && 'latest' || env.REL_CHANNEL }}


was this wrong before? if not why change over to REL_CHANNEL?

yea it was, we pull from the version channel in the bicep config so the uploads should also be to the channel tag

willdavsmith · 2024-07-11T16:22:19Z

.github/workflows/validate-bicep.yaml

@@ -37,12 +37,13 @@ jobs:
        uses: actions/checkout@v4
      - name: Parse release version and set environment variables
        run: python ./.github/scripts/get_release_version.py
-      - uses: oras-project/setup-oras@main
-      - name: Download rad-bicep
+      - name: Setup Azure CLI


why do we need azure CLI?

hmm we might not actually. will test to verify

willdavsmith · 2024-07-11T16:27:18Z

build/validate-bicep.sh

@@ -29,7 +30,7 @@ do
        exec 3>&-
    fi

-    if [[ ! $EXITCODE -eq 0 || ! -z $STDERR ]]
+    if [[ ! $EXITCODE -eq 0 || (! -z $STDERR && ! $STDERR == $WARNING_MSG* ) ]]


did you test this? I don't speak bash-fu

yea, ran this through a workflow_dispatch on the branch and it passes

Signed-off-by: sk593 <shruthikumar@microsoft.com>

codecov · 2024-07-19T17:17:58Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 61.00%. Comparing base (0cc2c64) to head (64bb10c).
Report is 1 commits behind head on feature-bicep-compiler-merge.

Additional details and impacted files

@@                      Coverage Diff                      @@
##           feature-bicep-compiler-merge    #7702   +/-   ##
=============================================================
  Coverage                         61.00%   61.00%           
=============================================================
  Files                               520      520           
  Lines                             27008    27008           
=============================================================
  Hits                              16476    16476           
  Misses                             9079     9079           
  Partials                           1453     1453

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: sk593 <shruthikumar@microsoft.com>

sk593 · 2024-07-22T19:10:45Z

.github/workflows/validate-bicep.yaml

@@ -33,17 +33,15 @@ jobs:
    name: Validate Bicep Code
    runs-on: ubuntu-latest
    steps:
+      - name: Setup and verify bicep CLI


there seems to be some lag between the bicep download and the verify step so moving this up to mitigate that

sk593 · 2024-07-22T19:11:01Z

.github/workflows/validate-bicep.yaml

      - name: Check out repo
        uses: actions/checkout@v4
-      - name: Parse release version and set environment variables


Don't think these variables were being used

Update validate bicep workflow  - This pull request fixes a bug in Radius and has an approved issue (issue link required). - This pull request adds or changes features of Radius and has an approved issue (issue link required). - This pull request is a minor refactor, code cleanup, test improvement, or other maintenance task and doesn't change the functionality of Radius (issue link optional).  Fixes: #issue_number --------- Signed-off-by: sk593 <shruthikumar@microsoft.com>

# Description This PR contains all the changes for the Bicep compiler merge. The branch has had smaller PRs opened for specific features that can be found below: - [x] Bicep types generator is updated to use the `bicep-types` repository as a submodule. This includes updating the types contract and adding support for resource methods (eg `listSecrets`): 43403fd . There was also a small update to how we handle flags on the `location` [property](19afe96) - [x] Update to build workflow to initialize the submodule and publish types to `biceptypes.azurecr.io` using the `bicep publish-extension` command: fed2462 - [x] Update validate bicep workflow to validate bicep files in the repository with the official Bicep compiler. This includes updating extension imports of all bicep files and adding a `bicepconfig.json` to the repository: #7702 - [x] Update functional test workflow to publish types to `biceptypes.azurecr.io`. This includes publishing the bicep types to a PR tag and adding a temporary `bicepconfig.json` to consume the PR-specific types [#7730] - [x] Update release workflow to publish types to a release-specific tag. This includes running a build in the `bicep-types-aws` repo to publish AWS types to a release tag - [x] Updates to `rad bicep download`. This command will download the official Bicep binary instead of the Radius one. The install script calls this command so the official Bicep binary will be downloaded on a Radius install: bb8da88 - [x] Updates to `rad init` to generate a default `bicepconfig.json`. This will pull the Radius and AWS types in our ACR with the current release tag: #7664 ## Type of change  - This pull request fixes a bug in Radius and has an approved issue (issue link required). - This pull request adds or changes features of Radius and has an approved issue (issue link required). - This pull request is a minor refactor, code cleanup, test improvement, or other maintenance task and doesn't change the functionality of Radius (issue link optional).  Fixes: #issue_number --------- # Description _Please explain the changes you've made._ ## Type of change  - This pull request fixes a bug in Radius and has an approved issue (issue link required). - This pull request adds or changes features of Radius and has an approved issue (issue link required). - This pull request is a minor refactor, code cleanup, test improvement, or other maintenance task and doesn't change the functionality of Radius (issue link optional).  Fixes: #issue_number --------- Signed-off-by: sk593 <shruthikumar@microsoft.com>

This PR contains all the changes for the Bicep compiler merge. The branch has had smaller PRs opened for specific features that can be found below: - [x] Bicep types generator is updated to use the `bicep-types` repository as a submodule. This includes updating the types contract and adding support for resource methods (eg `listSecrets`): radius-project@43403fd . There was also a small update to how we handle flags on the `location` [property](radius-project@19afe96) - [x] Update to build workflow to initialize the submodule and publish types to `biceptypes.azurecr.io` using the `bicep publish-extension` command: radius-project@fed2462 - [x] Update validate bicep workflow to validate bicep files in the repository with the official Bicep compiler. This includes updating extension imports of all bicep files and adding a `bicepconfig.json` to the repository: radius-project#7702 - [x] Update functional test workflow to publish types to `biceptypes.azurecr.io`. This includes publishing the bicep types to a PR tag and adding a temporary `bicepconfig.json` to consume the PR-specific types [radius-project#7730] - [x] Update release workflow to publish types to a release-specific tag. This includes running a build in the `bicep-types-aws` repo to publish AWS types to a release tag - [x] Updates to `rad bicep download`. This command will download the official Bicep binary instead of the Radius one. The install script calls this command so the official Bicep binary will be downloaded on a Radius install: radius-project@bb8da88 - [x] Updates to `rad init` to generate a default `bicepconfig.json`. This will pull the Radius and AWS types in our ACR with the current release tag: radius-project#7664  - This pull request fixes a bug in Radius and has an approved issue (issue link required). - This pull request adds or changes features of Radius and has an approved issue (issue link required). - This pull request is a minor refactor, code cleanup, test improvement, or other maintenance task and doesn't change the functionality of Radius (issue link optional).  Fixes: #issue_number --------- _Please explain the changes you've made._  - This pull request fixes a bug in Radius and has an approved issue (issue link required). - This pull request adds or changes features of Radius and has an approved issue (issue link required). - This pull request is a minor refactor, code cleanup, test improvement, or other maintenance task and doesn't change the functionality of Radius (issue link optional).  Fixes: #issue_number --------- Signed-off-by: sk593 <shruthikumar@microsoft.com>

# Description This PR contains all the changes for the Bicep compiler merge. The branch has had smaller PRs opened for specific features that can be found below: - [x] Bicep types generator is updated to use the `bicep-types` repository as a submodule. This includes updating the types contract and adding support for resource methods (eg `listSecrets`): radius-project@43403fd . There was also a small update to how we handle flags on the `location` [property](radius-project@19afe96) - [x] Update to build workflow to initialize the submodule and publish types to `biceptypes.azurecr.io` using the `bicep publish-extension` command: radius-project@fed2462 - [x] Update validate bicep workflow to validate bicep files in the repository with the official Bicep compiler. This includes updating extension imports of all bicep files and adding a `bicepconfig.json` to the repository: radius-project#7702 - [x] Update functional test workflow to publish types to `biceptypes.azurecr.io`. This includes publishing the bicep types to a PR tag and adding a temporary `bicepconfig.json` to consume the PR-specific types [radius-project#7730] - [x] Update release workflow to publish types to a release-specific tag. This includes running a build in the `bicep-types-aws` repo to publish AWS types to a release tag - [x] Updates to `rad bicep download`. This command will download the official Bicep binary instead of the Radius one. The install script calls this command so the official Bicep binary will be downloaded on a Radius install: radius-project@bb8da88 - [x] Updates to `rad init` to generate a default `bicepconfig.json`. This will pull the Radius and AWS types in our ACR with the current release tag: radius-project#7664 ## Type of change  - This pull request fixes a bug in Radius and has an approved issue (issue link required). - This pull request adds or changes features of Radius and has an approved issue (issue link required). - This pull request is a minor refactor, code cleanup, test improvement, or other maintenance task and doesn't change the functionality of Radius (issue link optional).  Fixes: #issue_number --------- # Description _Please explain the changes you've made._ ## Type of change  - This pull request fixes a bug in Radius and has an approved issue (issue link required). - This pull request adds or changes features of Radius and has an approved issue (issue link required). - This pull request is a minor refactor, code cleanup, test improvement, or other maintenance task and doesn't change the functionality of Radius (issue link optional).  Fixes: #issue_number --------- Signed-off-by: sk593 <shruthikumar@microsoft.com> Signed-off-by: Reshma Abdul Rahim <reshmarahim.abdul@microsoft.com>

This PR contains all the changes for the Bicep compiler merge. The branch has had smaller PRs opened for specific features that can be found below: - [x] Bicep types generator is updated to use the `bicep-types` repository as a submodule. This includes updating the types contract and adding support for resource methods (eg `listSecrets`): radius-project@43403fd . There was also a small update to how we handle flags on the `location` [property](radius-project@19afe96) - [x] Update to build workflow to initialize the submodule and publish types to `biceptypes.azurecr.io` using the `bicep publish-extension` command: radius-project@fed2462 - [x] Update validate bicep workflow to validate bicep files in the repository with the official Bicep compiler. This includes updating extension imports of all bicep files and adding a `bicepconfig.json` to the repository: radius-project#7702 - [x] Update functional test workflow to publish types to `biceptypes.azurecr.io`. This includes publishing the bicep types to a PR tag and adding a temporary `bicepconfig.json` to consume the PR-specific types [radius-project#7730] - [x] Update release workflow to publish types to a release-specific tag. This includes running a build in the `bicep-types-aws` repo to publish AWS types to a release tag - [x] Updates to `rad bicep download`. This command will download the official Bicep binary instead of the Radius one. The install script calls this command so the official Bicep binary will be downloaded on a Radius install: radius-project@bb8da88 - [x] Updates to `rad init` to generate a default `bicepconfig.json`. This will pull the Radius and AWS types in our ACR with the current release tag: radius-project#7664  - This pull request fixes a bug in Radius and has an approved issue (issue link required). - This pull request adds or changes features of Radius and has an approved issue (issue link required). - This pull request is a minor refactor, code cleanup, test improvement, or other maintenance task and doesn't change the functionality of Radius (issue link optional).  Fixes: #issue_number --------- _Please explain the changes you've made._  - This pull request fixes a bug in Radius and has an approved issue (issue link required). - This pull request adds or changes features of Radius and has an approved issue (issue link required). - This pull request is a minor refactor, code cleanup, test improvement, or other maintenance task and doesn't change the functionality of Radius (issue link optional).  Fixes: #issue_number --------- Signed-off-by: sk593 <shruthikumar@microsoft.com>

sk593 requested review from a team as code owners June 20, 2024 20:40

sk593 commented Jun 20, 2024

View reviewed changes

ytimocin reviewed Jun 21, 2024

View reviewed changes

rynowak reviewed Jun 24, 2024

View reviewed changes

sk593 force-pushed the update-validate-bicep branch 2 times, most recently from 5596c5c to 9c1a244 Compare June 26, 2024 23:24

ytimocin previously approved these changes Jul 5, 2024

View reviewed changes

sk593 force-pushed the feature-bicep-compiler-merge branch from fed2462 to 0cc2c64 Compare July 10, 2024 21:18

sk593 force-pushed the update-validate-bicep branch from 901d8c8 to d278a4a Compare July 10, 2024 21:22

sk593 added 4 commits July 10, 2024 14:22

update validate bicep workflow

70bd665

Signed-off-by: sk593 <shruthikumar@microsoft.com>

update val bicep workflow

5f90cd3

Signed-off-by: sk593 <shruthikumar@microsoft.com>

update aws imports

ede7664

Signed-off-by: sk593 <shruthikumar@microsoft.com>

update to extensions

d278a4a

Signed-off-by: sk593 <shruthikumar@microsoft.com>

willdavsmith approved these changes Jul 11, 2024

View reviewed changes

sk593 force-pushed the update-validate-bicep branch 4 times, most recently from 6f6797d to 68d53b3 Compare July 19, 2024 17:00

nit

68d53b3

Signed-off-by: sk593 <shruthikumar@microsoft.com>

sk593 force-pushed the update-validate-bicep branch from 64bb10c to 44e5b07 Compare July 22, 2024 16:04

update bicep download for correct version

1937334

Signed-off-by: sk593 <shruthikumar@microsoft.com>

sk593 force-pushed the update-validate-bicep branch from 44e5b07 to 1937334 Compare July 22, 2024 19:01

sk593 commented Jul 22, 2024

View reviewed changes

ytimocin approved these changes Jul 22, 2024

View reviewed changes

sk593 merged commit e34ecbc into feature-bicep-compiler-merge Jul 22, 2024
3 checks passed

sk593 deleted the update-validate-bicep branch July 22, 2024 19:59

sk593 mentioned this pull request Aug 1, 2024

Bicep compiler merge #7662

Merged

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update validate bicep workflow #7702

Update validate bicep workflow #7702

sk593 commented Jun 20, 2024

sk593 Jun 20, 2024 •

edited

Loading

sk593 Jun 20, 2024

ytimocin Jun 21, 2024

sk593 Jun 24, 2024

rynowak Jun 24, 2024

sk593 commented Jun 20, 2024 •

edited

Loading

ytimocin Jun 21, 2024

ytimocin Jun 21, 2024

ytimocin Jun 21, 2024

sk593 Jun 24, 2024

rynowak Jun 24, 2024

ytimocin Jun 21, 2024

rynowak Jun 24, 2024

sk593 Jun 24, 2024

rynowak Jun 24, 2024

rynowak Jun 24, 2024

sk593 Jun 26, 2024

rynowak Jun 26, 2024

sk593 Jul 1, 2024

github-actions bot commented Jun 26, 2024 •

edited

Loading

willdavsmith left a comment

willdavsmith Jul 11, 2024

sk593 Jul 11, 2024

willdavsmith Jul 11, 2024

sk593 Jul 11, 2024 •

edited

Loading

willdavsmith Jul 11, 2024

sk593 Jul 11, 2024

codecov bot commented Jul 19, 2024

sk593 Jul 22, 2024

sk593 Jul 22, 2024

Update validate bicep workflow #7702

Update validate bicep workflow #7702

Conversation

sk593 commented Jun 20, 2024

Description

Type of change

sk593 Jun 20, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sk593 commented Jun 20, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Jun 26, 2024 • edited Loading

Unit Tests

willdavsmith left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sk593 Jul 11, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Jul 19, 2024

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sk593 Jun 20, 2024 •

edited

Loading

sk593 commented Jun 20, 2024 •

edited

Loading

github-actions bot commented Jun 26, 2024 •

edited

Loading

sk593 Jul 11, 2024 •

edited

Loading