Envoyfilter with BOOTSTRAP configuration is not working with cluster created from service entry

[ukumar155]

Is this the right place to submit this?

• This is not a security vulnerability or a crashing bug
• This is not a question about how to use Istio

Bug Description

We are using REGISTRY_ONLY mode for our istio mesh.
Trying to deploy a bootstrap envoy filter with egress gateway, which is making an external call.

So we created a service entry:

apiVersion: networking.istio.io/v1beta1
kind: ServiceEntry
metadata:
 name: service_entry_httpbin
 namespace: istio-system
spec:
 hosts:
 - httpbin.org
 ports:
 - name: HTTP
   number: 80
   protocol: HTTP
 resolution: DNS

and envoyfilter in istio-system namespace:

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: bootstrap-config
  namespace: istio-system
spec:
  workloadSelector:
    labels:
      app: egress-gateway
  configPatches:            
  - applyTo: BOOTSTRAP
    patch:
      operation: MERGE
      value:
        bootstrap_extensions:
        - name: envoy.bootstrap.wasm
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.wasm.v3.WasmService
            singleton: true
            config:
              configuration:
                '@type': type.googleapis.com/google.protobuf.StringValue
                value:  |
                  {
                    "authority": "httpbin.org",
                    "cluster": "outbound|80||httpbin.org"
                  }
              vm_config:
                runtime: "envoy.wasm.runtime.v8"
                code:
                  local:
                    filename: "release.wasm"

But it did not worked and we can see 503 in gateway logs:

Status:503, Headers: 3, Body Size: 95, Trailers: 0, Context ID = 1
Body: upstream connect error or disconnect/reset before headers. reset reason: connection termination,

Then we removed service entry and created cluster from envoy filter and flow worked perfectly fine:

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: bootstrap-config
  namespace: istio-system
spec:
  workloadSelector:
    labels:
      app: egress-gateway
  configPatches:
  - applyTo: CLUSTER
    match:
      context: GATEWAY
    patch:
      operation: ADD
      value: # cluster specification
        name: "outbound|80||httpbin.org"
        type: STRICT_DNS
        connect_timeout: 2s
        lb_policy: ROUND_ROBIN
        dns_lookup_family: V4_ONLY
        load_assignment:
          cluster_name: outbound|80||httpbin.org
          endpoints:
            - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      protocol: TCP
                      address: httpbin.org
                      portValue: 80             
  - applyTo: BOOTSTRAP
    patch:
      operation: MERGE
      value:
        bootstrap_extensions:
        - name: envoy.bootstrap.wasm
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.wasm.v3.WasmService
            singleton: true
            config:
              configuration:
                '@type': type.googleapis.com/google.protobuf.StringValue
                value:  |
                  {
                    "authority": "httpbin.org",
                    "cluster": "outbound|80||httpbin.org"
                  }
              vm_config:
                runtime: "envoy.wasm.runtime.v8"
                code:
                  local:
                    filename: "release.wasm"

Is there a workaround using service entry??
@zirain @han

Version

$ istioctl version  
client version: 1.17.3
control plane version: 1.17.1
data plane version: 1.17.1
$ kubectl version
Client Version: v1.24.3
Kustomize Version: v4.5.4
Server Version: v1.25.16-eks-77b1e4e

Additional Information

No response

Affected product area

• Ambient
• Docs
• Dual Stack
• Installation
• Networking
• Performance and Scalability
• Extensions and Telemetry
• Security
• Test and Release
• User Experience
• Developer Infrastructure
• Upgrade
• Multi Cluster
• Virtual Machine
• Control Plane Revisions

[sai9920]

following need this for our case..

[zirain]

I recall there's a feature gate for that.
Anyway, this's removed in latest version, please use https://github.com/istio/istio/tree/master/samples/custom-bootstrap

[istio-policy-bot]

🚧 This issue or pull request has been closed due to not having had activity from an Istio team member since 2024-03-15. If you feel this issue or pull request deserves attention, please reopen the issue. Please see this wiki page for more information. Thank you for your contributions.

Created by the issue and PR lifecycle manager.