Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

WIP: recursive type unrolling prototype #4125

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from
Draft

WIP: recursive type unrolling prototype #4125

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
306ec64
Update to pre-release bridge
t0yv0 Jun 27, 2024
911f8ad
New bridge version and applying unrec.SimplifyRecursiveTypes
t0yv0 Jun 27, 2024
6e2e292
Try recursive unrolling for Node
t0yv0 Jun 27, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions examples/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ require (
github.com/aws/aws-sdk-go-v2/service/iam v1.33.1
github.com/pulumi/providertest v0.0.11
github.com/pulumi/pulumi-aws/provider/v6 v6.0.0-00010101000000-000000000000
github.com/pulumi/pulumi-terraform-bridge/pf v0.38.0
github.com/pulumi/pulumi-terraform-bridge/pf v0.38.1-0.20240627164523-242339028d5a
github.com/pulumi/pulumi-terraform-bridge/testing v0.0.2-0.20230927165309-e3fd9503f2d3
github.com/pulumi/pulumi/pkg/v3 v3.121.0
github.com/stretchr/testify v1.9.0
Expand Down Expand Up @@ -373,7 +373,7 @@ require (
github.com/pulumi/appdash v0.0.0-20231130102222-75f619a67231 // indirect
github.com/pulumi/esc v0.9.1 // indirect
github.com/pulumi/inflector v0.1.1 // indirect
github.com/pulumi/pulumi-terraform-bridge/v3 v3.85.0 // indirect
github.com/pulumi/pulumi-terraform-bridge/v3 v3.85.1-0.20240627164523-242339028d5a // indirect
github.com/pulumi/pulumi-terraform-bridge/x/muxer v0.0.8 // indirect
github.com/pulumi/pulumi/sdk/v3 v3.121.0 // indirect
github.com/pulumi/terraform-diff-reader v0.0.2 // indirect
Expand Down
8 changes: 4 additions & 4 deletions examples/go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2315,12 +2315,12 @@ github.com/pulumi/inflector v0.1.1 h1:dvlxlWtXwOJTUUtcYDvwnl6Mpg33prhK+7mzeF+Sob
github.com/pulumi/inflector v0.1.1/go.mod h1:HUFCjcPTz96YtTuUlwG3i3EZG4WlniBvR9bd+iJxCUY=
github.com/pulumi/providertest v0.0.11 h1:mg8MQ7Cq7+9XlHIkBD+aCqQO4mwAJEISngZgVdnQUe8=
github.com/pulumi/providertest v0.0.11/go.mod h1:HsxjVsytcMIuNj19w1lT2W0QXY0oReXl1+h6eD2JXP8=
github.com/pulumi/pulumi-terraform-bridge/pf v0.38.0 h1:0+A+ZkoZWy5EOd4zcnM7tjoQ4V1jV/koR8YvWJ8TK/E=
github.com/pulumi/pulumi-terraform-bridge/pf v0.38.0/go.mod h1:JGOlvwSWY+jEt1V9sI/L8HAP9DBr74aXD10oi5nUJaI=
github.com/pulumi/pulumi-terraform-bridge/pf v0.38.1-0.20240627164523-242339028d5a h1:/qeuaUIEyEBh24KbALt0gk+9BzpxxrQxYt8f3RH2o/4=
github.com/pulumi/pulumi-terraform-bridge/pf v0.38.1-0.20240627164523-242339028d5a/go.mod h1:JGOlvwSWY+jEt1V9sI/L8HAP9DBr74aXD10oi5nUJaI=
github.com/pulumi/pulumi-terraform-bridge/testing v0.0.2-0.20230927165309-e3fd9503f2d3 h1:bBWWeAtSPPYpKYlPZr2h0BiYgWQpHRIk0HO/MQmB+jc=
github.com/pulumi/pulumi-terraform-bridge/testing v0.0.2-0.20230927165309-e3fd9503f2d3/go.mod h1:vAQ7DeddebQ7FHdRaSG6ijuS28FS9PC4j8Y9wUuue0c=
github.com/pulumi/pulumi-terraform-bridge/v3 v3.85.0 h1:Zv6OPQdkGERufe2Mq9D92xbTm5mg3uhllh0ryrcrrds=
github.com/pulumi/pulumi-terraform-bridge/v3 v3.85.0/go.mod h1:a7t2qe4smtB7HlbHlelQxjJQn8DFNB3Gbe5Ot2W7GZU=
github.com/pulumi/pulumi-terraform-bridge/v3 v3.85.1-0.20240627164523-242339028d5a h1:aJqL7JhQWc8FN6CZ2fGyIBDBbJ0olMrnxWK8FzYIpYg=
github.com/pulumi/pulumi-terraform-bridge/v3 v3.85.1-0.20240627164523-242339028d5a/go.mod h1:a7t2qe4smtB7HlbHlelQxjJQn8DFNB3Gbe5Ot2W7GZU=
github.com/pulumi/pulumi-terraform-bridge/x/muxer v0.0.8 h1:mav2tSitA9BPJPLLahKgepHyYsMzwaTm4cvp0dcTMYw=
github.com/pulumi/pulumi-terraform-bridge/x/muxer v0.0.8/go.mod h1:qUYk2c9i/yqMGNj9/bQyXpS39BxNDSXYjVN1njnq0zY=
github.com/pulumi/pulumi/pkg/v3 v3.121.0 h1:cLUQJYGJKfgCY0ubJo8dVwmsIm2WcgTprb9Orc/yiFg=
Expand Down
2 changes: 1 addition & 1 deletion provider/cmd/pulumi-resource-aws/bridge-metadata.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -232628,4 +232628,4 @@
"aws:workspaces/getWorkspace:getWorkspace": 0
}
}
}
}
26,241 changes: 7,311 additions & 18,930 deletions provider/cmd/pulumi-resource-aws/schema.json
View file
Open in desktop

Large diffs are not rendered by default.

81 changes: 4 additions & 77 deletions provider/cmd/pulumi-tfgen-aws/wafv2.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,10 @@
package main

import (
"fmt"
"strings"
"context"

"github.com/pulumi/pulumi-terraform-bridge/v3/pkg/tfgen/unrec"
"github.com/pulumi/pulumi/pkg/v3/codegen/schema"
"github.com/pulumi/pulumi/sdk/v3/go/common/util/contract"
)

// WafV2 has two problematic resources with massive schemas.
Expand All @@ -19,79 +18,7 @@ import (
// to find all references to all Statement types and replace those
// references with a reference to the single top-level Statement types.
func replaceWafV2TypesWithRecursive(pulumiPackageSpec *schema.PackageSpec) {
rootStatementTypes := []string{"RuleGroupRuleStatement", "WebAclRuleStatement"}

// We'll collect all referenced types that we replace with the root
// type, so that we could remove all of elided types and their children.
var elidedRefs []string
for tok, ts := range pulumiPackageSpec.Types {
// Skip everything except WafV2 types.
if !strings.Contains(tok, ":wafv2/") {
continue
}
// The recursive structures look like these currently:
// RuleStatement has a few properties like
// AndStatement, OrStatement, NotStatement, all of separate types.
// Each of those types has a property `statements` that point
// to the next layer of statement types, but should actually point
// to the top one recursively.
// So, we find all the `statements` properties (continue if not found).
var oldRef string
if prop, has := ts.Properties["statements"]; has {
contract.Assertf(prop.TypeSpec.Items != nil, "statements property must be an array")
oldRef = prop.TypeSpec.Items.Ref
} else {
continue
}
// Add the currently referenced type to the list to be elided.
// Example of a reference:
// #/types/aws:wafv2/RuleGroupRuleStatement:RuleGroupRuleStatement
refType := strings.Split(oldRef, ":")[2]
elidedRefs = append(elidedRefs, refType)
// Get the current type name.
typeName := strings.Split(tok, ":")[2]
for _, rule := range rootStatementTypes {
if !strings.HasPrefix(typeName, rule) {
continue
}
// Build a reference to the root RuleStatement type and replace the property.
ref := fmt.Sprintf("#/types/aws:wafv2/%s:%[1]s", rule)
ts.Properties["statements"] = schema.PropertySpec{
Description: "The statements to combine.",
TypeSpec: schema.TypeSpec{
Type: "array",
Items: &schema.TypeSpec{
Ref: ref,
},
},
}
}
}

// We collected a list of types `elidedRefs` that used to be referenced
// by other types, but aren't anymore because we replaced them with
// top-level references.
// We want to remove all those types from the schema, and also we want
// to remove all their subtypes. TF types are unidirectional, and our
// naming is very predictable, so all the subtypes' names start with
// the parent type name.
// Loop through all types again and collect the ones that start with
// one of the elided reference type.
var elidedTypes []string
for tok := range pulumiPackageSpec.Types {
if !strings.Contains(tok, "wafv2") {
continue
}
for _, ref := range elidedRefs {
if strings.Contains(tok, ref) {
elidedTypes = append(elidedTypes, tok)
break
}
}
}

// Now remove all elided types from the schema.
for _, tok := range elidedTypes {
delete(pulumiPackageSpec.Types, tok)
if err := unrec.SimplifyRecursiveTypes(context.Background(), pulumiPackageSpec); err != nil {
panic(err)
}
}
4 changes: 2 additions & 2 deletions provider/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@ require (
github.com/hashicorp/terraform-provider-aws v1.60.1-0.20220923175450-ca71523cdc36
github.com/mitchellh/go-homedir v1.1.0
github.com/pulumi/providertest v0.0.11
github.com/pulumi/pulumi-terraform-bridge/pf v0.38.0
github.com/pulumi/pulumi-terraform-bridge/v3 v3.85.0
github.com/pulumi/pulumi-terraform-bridge/pf v0.38.1-0.20240627164523-242339028d5a
github.com/pulumi/pulumi-terraform-bridge/v3 v3.85.1-0.20240627164523-242339028d5a
github.com/pulumi/pulumi/pkg/v3 v3.121.0
github.com/pulumi/pulumi/sdk/v3 v3.121.0
github.com/stretchr/testify v1.9.0
Expand Down
8 changes: 4 additions & 4 deletions provider/go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2331,10 +2331,10 @@ github.com/pulumi/providertest v0.0.11 h1:mg8MQ7Cq7+9XlHIkBD+aCqQO4mwAJEISngZgVd
github.com/pulumi/providertest v0.0.11/go.mod h1:HsxjVsytcMIuNj19w1lT2W0QXY0oReXl1+h6eD2JXP8=
github.com/pulumi/pulumi-java/pkg v0.11.0 h1:Jw9gBvyfmfOMq/EkYDm9+zGPxsDAA8jfeMpHmtZ+1oA=
github.com/pulumi/pulumi-java/pkg v0.11.0/go.mod h1:sXAk25P47AQVQL6ilAbFmRNgZykC7og/+87ihnqzFTc=
github.com/pulumi/pulumi-terraform-bridge/pf v0.38.0 h1:0+A+ZkoZWy5EOd4zcnM7tjoQ4V1jV/koR8YvWJ8TK/E=
github.com/pulumi/pulumi-terraform-bridge/pf v0.38.0/go.mod h1:JGOlvwSWY+jEt1V9sI/L8HAP9DBr74aXD10oi5nUJaI=
github.com/pulumi/pulumi-terraform-bridge/v3 v3.85.0 h1:Zv6OPQdkGERufe2Mq9D92xbTm5mg3uhllh0ryrcrrds=
github.com/pulumi/pulumi-terraform-bridge/v3 v3.85.0/go.mod h1:a7t2qe4smtB7HlbHlelQxjJQn8DFNB3Gbe5Ot2W7GZU=
github.com/pulumi/pulumi-terraform-bridge/pf v0.38.1-0.20240627164523-242339028d5a h1:/qeuaUIEyEBh24KbALt0gk+9BzpxxrQxYt8f3RH2o/4=
github.com/pulumi/pulumi-terraform-bridge/pf v0.38.1-0.20240627164523-242339028d5a/go.mod h1:JGOlvwSWY+jEt1V9sI/L8HAP9DBr74aXD10oi5nUJaI=
github.com/pulumi/pulumi-terraform-bridge/v3 v3.85.1-0.20240627164523-242339028d5a h1:aJqL7JhQWc8FN6CZ2fGyIBDBbJ0olMrnxWK8FzYIpYg=
github.com/pulumi/pulumi-terraform-bridge/v3 v3.85.1-0.20240627164523-242339028d5a/go.mod h1:a7t2qe4smtB7HlbHlelQxjJQn8DFNB3Gbe5Ot2W7GZU=
github.com/pulumi/pulumi-terraform-bridge/x/muxer v0.0.8 h1:mav2tSitA9BPJPLLahKgepHyYsMzwaTm4cvp0dcTMYw=
github.com/pulumi/pulumi-terraform-bridge/x/muxer v0.0.8/go.mod h1:qUYk2c9i/yqMGNj9/bQyXpS39BxNDSXYjVN1njnq0zY=
github.com/pulumi/pulumi-yaml v1.8.0 h1:bhmidiCMMuzsJao5FE0UR69iF3WVKPCFrRkzjotFNn4=
Expand Down
Loading
Loading