bug(apply): unable to apply K8s CRD using recipe

[mittachaitu]

Description

Unable to deploy CRDs by using recipe concept. I applied the following
mayastor-recipe.txt to deploy mayastor and faced the following error.

/usr/local/go/src/runtime/asm_amd64.s:1357
I1214 15:05:40.608218       1 recipe.go:660] Will execute: Recipe "kubera" / "default-mayastor-install": Status "Error" "Task failed: Index 5: Name \"apply-mayastorpool-crd\": Recipe \"kubera\" / \"default-mayastor-install\": cannot convert int64 to float64"
E1214 15:05:41.615086       1 reconciler.go:118] Reconcile failed: Controller "sync-recipe": Name "kubera" / "default-mayastor-install": Error cannot convert int64 to float64
Task failed: Index 5: Name "apply-mayastorpool-crd": Recipe "kubera" / "default-mayastor-install"
mayadata.io/d-operators/pkg/recipe.(*Runner).runAllTasks
	/mayadata.io/d-operators/pkg/recipe/recipe.go:584
mayadata.io/d-operators/pkg/recipe.(*Runner).Run
	/mayadata.io/d-operators/pkg/recipe/recipe.go:757
mayadata.io/d-operators/controller/recipe.(*Reconciler).invoke
	/mayadata.io/d-operators/controller/recipe/reconciler.go:72
mayadata.io/d-operators/common/controller.(*Reconciler).Reconcile
	/mayadata.io/d-operators/common/controller/reconciler.go:153
mayadata.io/d-operators/controller/recipe.Sync
	/mayadata.io/d-operators/controller/recipe/reconciler.go:170
openebs.io/metac/controller/generic.(*InlineHookInvoker).Invoke
	/go/pkg/mod/github.com/!amit!kumar!das/metac@v0.4.0/controller/generic/inlinehook.go:68
openebs.io/metac/controller/generic.(*HookInvoker).Invoke

• D-Operators internally converting unstructured into concrete Go type if
  the resource is CustomResourceDefination due to this conversion from int64 to float64 is failing.

Expected Behavior

• Recipe should be applied without any errors.

Current Behavior

• Unable to deploy the Mayastor using the recipe.

Possible Solution

• IMO D-Operators should deal with map[string]interface{} instead of
  converting into concrete type.

Steps To Reproduce

• Apply the above YAML by deploying d-operators.
  (Or)
• Run conversion.go](https://github.com/mayadata-io/d-operators/files/5707475/conversion.txt) in the go environment.

[AmitKumarDas]

Following yaml when applied via Recipe code should result in error:

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: mayastorpools.openebs.io
spec:
  group: openebs.io
  versions:
    - name: v1alpha1
      served: true
      storage: true
      subresources:
        status: {}
      schema:
        openAPIV3Schema:
          type: object
          properties:
            apiVersion:
              type: string
            kind:
              type: string
            metadata:
              type: object
            spec:
              description: Specification of the mayastor pool.
              type: object
              required:
              - node
              - disks
              properties:
                node:
                  description: Name of the k8s node where the storage pool is located.
                  type: string
                disks:
                  description: Disk devices (paths or URIs) that should be used for the pool.
                  type: array
                  items:
                    type: string
            status:
              description: Status part updated by the pool controller.
              type: object
              properties:
                state:
                  description: Pool state.
                  type: string
                reason:
                  description: Reason for the pool state value if applicable.
                  type: string
                disks:
                  description: Disk device URIs that are actually used for the pool.
                  type: array
                  items:
                    type: string
                capacity:
                  description: Capacity of the pool in bytes.
                  type: integer
                  format: int64
                  minimum: 0
                used:
                  description: How many bytes are used in the pool.
                  type: integer
                  format: int64
                  minimum: 0
      additionalPrinterColumns:
      - name: Node
        type: string
        description: Node where the storage pool is located
        jsonPath: .spec.node
      - name: State
        type: string
        description: State of the storage pool
        jsonPath: .status.state
      - name: Age
        type: date
        jsonPath: .metadata.creationTimestamp
  scope: Namespaced
  names:
    kind: MayastorPool
    listKind: MayastorPoolList
    plural: mayastorpools
    singular: mayastorpool
    shortNames: ["msp"]

[AmitKumarDas]

Errors while unmarshaling yaml string to unstructured & CRD type

Error: Failed to unmarshal: : error converting YAML to JSON: yaml: line 7: found character that cannot start any token
Remedy: remove tab spacing for 4 single spaces
NOTE: This had to be done throughout either manually or via some tool

Error: Failed to convert to CRD type: cannot convert int64 to float64
Note: There is no information on the line or field that is leading to this error
Note: YAML string to unstructured was done via github.com/ghodss/yaml
Note: ghodss/yaml lib does not take into account unstructured datatypes supported by K8s lib
e.g. K8s supports int64 instead of int
- If unstructured makes use of 0 then it is considered as int 
- int is an invalid type as per k8s lib
- Right way is to typecast above to int64(0)

Errors applying (create & update) unstructured instances to k8s cluster

In this case, string boolean was used instead of bool & hence below error

"versions": []interface{}{
   map[string]interface{}{
     "name":    "v1alpha1",
     "served":  "true", // error
     "storage": true, // ok
  },
},

--- FAIL: TestCRDApply (1.03s)
    crd_int_test.go:171: Error while testing: CustomResourceDefinition in version "v1" 
cannot be handled as a CustomResourceDefinition: v1.CustomResourceDefinition.Spec: 
v1.CustomResourceDefinitionSpec.Versions: []v1.CustomResourceDefinitionVersion: 
v1.CustomResourceDefinitionVersion.Served: ReadBool: expect t or f, but found ", 
error found in #10 byte of ...|"served":"true","sto|..., bigger context 
...|"}},"type":"object"}},"type":"object"}},"served":"true","storage":true,"subresources":{"status":
{}}}|...: %!s(*types.ApplyResult=<nil>)
FAIL

Conclusion

• There will be issues if logic
  • Converts a yaml string to unstructured via non k8s lib &
  • Then to a typed instance via k8s lib
• There will be issues if logic
  • Converts a yaml string to unstructured via non k8s lib e.g. ghodss/yaml &
  • Then back to bytes/string via k8s lib
• It is best to avoid use of yaml string wherever possible
• It is good to use unstructured instances directly
  • Since it provides better error information
  • It provides exact field name that has error(s)
• Alternatively, need to look into the updated versions & libraries that k8s use internally to fix these kind of issues.