Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gRPC Example hang #99

Closed
jeanlucmongrain opened this issue May 30, 2018 · 8 comments
Closed

gRPC Example hang #99

jeanlucmongrain opened this issue May 30, 2018 · 8 comments

Comments

@jeanlucmongrain
Copy link

jeanlucmongrain commented May 30, 2018

Trying https://github.com/solo-io/gloo/tree/master/example/grpc
with latest version of Virtualbox, Minikube, kubectl, glooctl
on a blank minikube.

at this step:

[07:01] curl $GRPC_URL/bookstore.Bookstore/ListShelves
curl: (7) Failed to connect to 10.0.2.15 port 30508: Connection timed out
[07:06] 

Logs

Bookstore

2018/05/30 06:57:59 listening on 8080
2018/05/30 06:58:15 /grpc.reflection.v1alpha.ServerReflection/ServerReflectionInfo

kube-ingress

ERROR: logging before flag.Parse: W0530 06:46:01.324885       1 client_config.go:529] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
ERROR: logging before flag.Parse: W0530 06:46:01.326152       1 client_config.go:529] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
"Wed, 30 May 2018 06:46:01 UTC: github.com/solo-io/gloo/pkg/storage/crd/crd_storage_client.go:103"	registering crd v1.crd{
  Plural:    "upstreams",
  Group:     "gloo.solo.io",
  Version:   "v1",
  Kind:      "Upstream",
  ShortName: "us",
}
"Wed, 30 May 2018 06:46:01 UTC: github.com/solo-io/gloo/pkg/storage/crd/crd_storage_client.go:103"	registering crd v1.crd{
  Plural:    "virtualservices",
  Group:     "gloo.solo.io",
  Version:   "v1",
  Kind:      "VirtualService",
  ShortName: "vs",
}
"Wed, 30 May 2018 06:46:01 UTC: github.com/solo-io/gloo/pkg/storage/crd/crd_storage_client.go:103"	registering crd v1.crd{
  Plural:    "virtualmeshes",
  Group:     "gloo.solo.io",
  Version:   "v1",
  Kind:      "VirtualMesh",
  ShortName: "vm",
}
"Wed, 30 May 2018 06:46:01 UTC: github.com/solo-io/gloo/cmd/kube-ingress-controller/main.go:105"	starting ingress controller
"Wed, 30 May 2018 06:46:01 UTC: github.com/solo-io/gloo/vendor/github.com/solo-io/kubecontroller/controller.go:113"	Starting "gloo-ingress-syncer" controller
"Wed, 30 May 2018 06:46:01 UTC: github.com/solo-io/gloo/vendor/github.com/solo-io/kubecontroller/controller.go:116"	Waiting for informer caches to sync
"Wed, 30 May 2018 06:46:01 UTC: github.com/solo-io/gloo/cmd/kube-ingress-controller/main.go:81"	starting ingress status sync
"Wed, 30 May 2018 06:46:01 UTC: github.com/solo-io/gloo/cmd/kube-ingress-controller/main.go:94"	starting ingress sync
"Wed, 30 May 2018 06:46:01 UTC: github.com/solo-io/gloo/vendor/github.com/solo-io/kubecontroller/controller.go:113"	Starting "kube-ingress-controller" controller
"Wed, 30 May 2018 06:46:01 UTC: github.com/solo-io/gloo/vendor/github.com/solo-io/kubecontroller/controller.go:116"	Waiting for informer caches to sync
"Wed, 30 May 2018 06:46:01 UTC: github.com/solo-io/gloo/vendor/github.com/solo-io/kubecontroller/controller.go:121"	Starting workers
"Wed, 30 May 2018 06:46:01 UTC: github.com/solo-io/gloo/vendor/github.com/solo-io/kubecontroller/controller.go:127"	Started workers
"Wed, 30 May 2018 06:46:01 UTC: github.com/solo-io/gloo/vendor/github.com/solo-io/kubecontroller/controller.go:121"	Starting workers
"Wed, 30 May 2018 06:46:01 UTC: github.com/solo-io/gloo/vendor/github.com/solo-io/kubecontroller/controller.go:127"	Started workers

control-plane

non stop flow of:

ERROR: logging before flag.Parse: E0530 07:11:03.617946       1 reflector.go:205] github.com/solo-io/gloo/pkg/storage/crd/roles.go:84: Failed to list *v1.Role: roles.gloo.solo.io is forbidden: User "system:serviceaccount:gloo-system:default" cannot list roles.gloo.solo.io in the namespace "gloo-system"
ERROR: logging before flag.Parse: E0530 07:11:04.620914       1 reflector.go:205] github.com/solo-io/gloo/pkg/storage/crd/roles.go:84: Failed to list *v1.Role: roles.gloo.solo.io is forbidden: User "system:serviceaccount:gloo-system:default" cannot list roles.gloo.solo.io in the namespace "gloo-system"
ERROR: logging before flag.Parse: E0530 07:11:05.623252       1 reflector.go:205] github.com/solo-io/gloo/pkg/storage/crd/roles.go:84: Failed to list *v1.Role: roles.gloo.solo.io is forbidden: User "system:serviceaccount:gloo-system:default" cannot list roles.gloo.solo.io in the namespace "gloo-system"
ERROR: logging before flag.Parse: E0530 07:11:06.626267       1 reflector.go:205] github.com/solo-io/gloo/pkg/storage/crd/roles.go:84: Failed to list *v1.Role: roles.gloo.solo.io is forbidden: User "system:serviceaccount:gloo-system:default" cannot list roles.gloo.solo.io in the namespace "gloo-system"
ERROR: logging before flag.Parse: E0530 07:11:07.628632       1 reflector.go:205] github.com/solo-io/gloo/pkg/storage/crd/roles.go:84: Failed to list *v1.Role: roles.gloo.solo.io is forbidden: User "system:serviceaccount:gloo-system:default" cannot list roles.gloo.solo.io in the namespace "gloo-system"
ERROR: logging before flag.Parse: E0530 07:11:08.633828       1 reflector.go:205] github.com/solo-io/gloo/pkg/storage/crd/roles.go:84: Failed to list *v1.Role: roles.gloo.solo.io is forbidden: User "system:serviceaccount:gloo-system:default" cannot list roles.gloo.solo.io in the namespace "gloo-system"
ERROR: logging before flag.Parse: E0530 07:11:09.650339       1 reflector.go:205] github.com/solo-io/gloo/pkg/storage/crd/roles.go:84: Failed to list *v1.Role: roles.gloo.solo.io is forbidden: User "system:serviceaccount:gloo-system:default" cannot list roles.gloo.solo.io in the namespace "gloo-system"
ERROR: logging before flag.Parse: E0530 07:11:10.653396       1 reflector.go:205] github.com/solo-io/gloo/pkg/storage/crd/roles.go:84: Failed to list *v1.Role: roles.gloo.solo.io is forbidden: User "system:serviceaccount:gloo-system:default" cannot list roles.gloo.solo.io in the namespace "gloo-system"
ERROR: logging before flag.Parse: E0530 07:11:11.655923       1 reflector.go:205] github.com/solo-io/gloo/pkg/storage/crd/roles.go:84: Failed to list *v1.Role: roles.gloo.solo.io is forbidden: User "system:serviceaccount:gloo-system:default" cannot list roles.gloo.solo.io in the namespace "gloo-system"

ingress

[2018-05-30 06:46:50.281][1][info][main] external/envoy/source/server/server.cc:200] initializing epoch 0 (hot restart version=9.200.16384.127.options=capacity=16384, num_slots=8209 hash=228984379728933363)
[2018-05-30 06:46:50.281][1][info][main] external/envoy/source/server/server.cc:202] statically linked extensions:
[2018-05-30 06:46:50.281][1][info][main] external/envoy/source/server/server.cc:204]   access_loggers: envoy.file_access_log,envoy.http_grpc_access_log
[2018-05-30 06:46:50.281][1][info][main] external/envoy/source/server/server.cc:207]   filters.http: envoy.buffer,envoy.cors,envoy.ext_authz,envoy.fault,envoy.grpc_http1_bridge,envoy.grpc_json_transcoder,envoy.grpc_web,envoy.gzip,envoy.health_check,envoy.http_dynamo_filter,envoy.ip_tagging,envoy.lua,envoy.rate_limit,envoy.router,envoy.squash,io.solo.azure_functions,io.solo.function_router,io.solo.gcloudfunc,io.solo.lambda,io.solo.nats_streaming,io.solo.transformation
[2018-05-30 06:46:50.281][1][info][main] external/envoy/source/server/server.cc:210]   filters.listener: envoy.listener.original_dst,envoy.listener.proxy_protocol
[2018-05-30 06:46:50.281][1][info][main] external/envoy/source/server/server.cc:213]   filters.network: envoy.client_ssl_auth,envoy.echo,envoy.ext_authz,envoy.http_connection_manager,envoy.mongo_proxy,envoy.ratelimit,envoy.redis_proxy,envoy.tcp_proxy
[2018-05-30 06:46:50.281][1][info][main] external/envoy/source/server/server.cc:215]   stat_sinks: envoy.dog_statsd,envoy.metrics_service,envoy.statsd
[2018-05-30 06:46:50.281][1][info][main] external/envoy/source/server/server.cc:217]   tracers: envoy.dynamic.ot,envoy.lightstep,envoy.zipkin
[2018-05-30 06:46:50.281][1][info][main] external/envoy/source/server/server.cc:220]   transport_sockets.downstream: raw_buffer,ssl
[2018-05-30 06:46:50.281][1][info][main] external/envoy/source/server/server.cc:223]   transport_sockets.upstream: raw_buffer,ssl
[2018-05-30 06:46:50.294][1][warning][upstream] external/envoy/source/common/config/grpc_mux_impl.cc:224] gRPC config stream closed: 1, 
[2018-05-30 06:46:50.294][1][warning][upstream] external/envoy/source/common/config/grpc_mux_impl.cc:39] Unable to establish new stream
[2018-05-30 06:46:50.294][1][info][config] external/envoy/source/server/configuration_impl.cc:52] loading 0 listener(s)
[2018-05-30 06:46:50.294][1][info][config] external/envoy/source/server/configuration_impl.cc:92] loading tracing configuration
[2018-05-30 06:46:50.294][1][info][config] external/envoy/source/server/configuration_impl.cc:114] loading stats sink configuration
[2018-05-30 06:46:50.294][1][info][main] external/envoy/source/server/server.cc:399] starting main dispatch loop
[2018-05-30 06:46:55.295][1][warning][upstream] external/envoy/source/common/config/grpc_mux_impl.cc:224] gRPC config stream closed: 1, 
[2018-05-30 06:46:55.296][1][warning][upstream] external/envoy/source/common/config/grpc_mux_impl.cc:39] Unable to establish new stream
[2018-05-30 06:47:00.297][1][warning][upstream] external/envoy/source/common/config/grpc_mux_impl.cc:224] gRPC config stream closed: 1, 
[2018-05-30 06:47:00.297][1][warning][upstream] external/envoy/source/common/config/grpc_mux_impl.cc:39] Unable to establish new stream
[2018-05-30 06:47:05.297][1][warning][upstream] external/envoy/source/common/config/grpc_mux_impl.cc:224] gRPC config stream closed: 1, 
[2018-05-30 06:47:05.297][1][warning][upstream] external/envoy/source/common/config/grpc_mux_impl.cc:39] Unable to establish new stream
[2018-05-30 06:47:10.298][1][warning][upstream] external/envoy/source/common/config/grpc_mux_impl.cc:224] gRPC config stream closed: 1, 
[2018-05-30 06:47:10.298][1][warning][upstream] external/envoy/source/common/config/grpc_mux_impl.cc:39] Unable to establish new stream
[2018-05-30 06:47:15.297][1][warning][upstream] external/envoy/source/common/config/grpc_mux_impl.cc:224] gRPC config stream closed: 1, 
[2018-05-30 06:47:15.298][1][warning][upstream] external/envoy/source/common/config/grpc_mux_impl.cc:39] Unable to establish new stream
[2018-05-30 06:47:20.298][1][warning][upstream] external/envoy/source/common/config/grpc_mux_impl.cc:224] gRPC config stream closed: 1, 
[2018-05-30 06:47:20.298][1][warning][upstream] external/envoy/source/common/config/grpc_mux_impl.cc:39] Unable to establish new stream
[2018-05-30 06:47:25.297][1][warning][upstream] external/envoy/source/common/config/grpc_mux_impl.cc:224] gRPC config stream closed: 1, 
[2018-05-30 06:47:25.298][1][warning][upstream] external/envoy/source/common/config/grpc_mux_impl.cc:39] Unable to establish new stream
[2018-05-30 06:47:30.297][1][warning][upstream] external/envoy/source/common/config/grpc_mux_impl.cc:224] gRPC config stream closed: 1, 
[2018-05-30 06:47:30.297][1][warning][upstream] external/envoy/source/common/config/grpc_mux_impl.cc:39] Unable to establish new stream
[2018-05-30 06:47:35.299][1][warning][upstream] external/envoy/source/common/config/grpc_mux_impl.cc:224] gRPC config stream closed: 1, 
[2018-05-30 06:47:35.299][1][warning][upstream] external/envoy/source/common/config/grpc_mux_impl.cc:39] Unable to establish new stream
[2018-05-30 06:47:40.298][1][warning][upstream] external/envoy/source/common/config/grpc_mux_impl.cc:224] gRPC config stream closed: 1, 
[2018-05-30 06:47:40.298][1][warning][upstream] external/envoy/source/common/config/grpc_mux_impl.cc:39] Unable to establish new stream
[2018-05-30 06:47:45.299][1][warning][upstream] external/envoy/source/common/config/grpc_mux_impl.cc:224] gRPC config stream closed: 1, 
[2018-05-30 06:47:45.299][1][warning][upstream] external/envoy/source/common/config/grpc_mux_impl.cc:39] Unable to establish new stream
[2018-05-30 06:47:47.799][1][info][upstream] external/envoy/source/common/upstream/cluster_manager_impl.cc:127] cm init: initializing cds
[2018-05-30 06:47:50.313][1][info][upstream] external/envoy/source/common/upstream/cluster_manager_impl.cc:379] add/update cluster gloo-system-control-plane-8081 during init
[2018-05-30 06:47:50.317][1][info][upstream] external/envoy/source/common/upstream/cluster_manager_impl.cc:379] add/update cluster gloo-system-ingress-8080 during init
[2018-05-30 06:47:50.320][1][info][upstream] external/envoy/source/common/upstream/cluster_manager_impl.cc:379] add/update cluster gloo-system-ingress-8443 during init
[2018-05-30 06:47:50.321][1][info][upstream] external/envoy/source/common/upstream/cluster_manager_impl.cc:108] cm init: initializing secondary clusters
[2018-05-30 06:47:50.324][1][info][upstream] external/envoy/source/common/upstream/cluster_manager_impl.cc:131] cm init: all clusters initialized
[2018-05-30 06:47:50.324][1][info][main] external/envoy/source/server/server.cc:383] all clusters initialized. initializing init manager
[2018-05-30 06:47:50.325][1][info][config] external/envoy/source/server/listener_manager_impl.cc:602] all dependencies initialized. starting workers
[2018-05-30 06:57:32.189][1][info][upstream] external/envoy/source/common/upstream/cluster_manager_impl.cc:385] add/update cluster default-grpc-bookstore-8080 starting warming
[2018-05-30 06:57:32.189][1][info][upstream] external/envoy/source/common/upstream/cluster_manager_impl.cc:392] warming cluster default-grpc-bookstore-8080 complete
[2018-05-30 06:58:15.764][1][info][upstream] external/envoy/source/common/upstream/cluster_manager_impl.cc:385] add/update cluster default-grpc-bookstore-8080 starting warming
[2018-05-30 06:58:15.764][1][info][upstream] external/envoy/source/common/upstream/cluster_manager_impl.cc:392] warming cluster default-grpc-bookstore-8080 complete
[2018-05-30 07:02:50.326][1][info][main] external/envoy/source/server/drain_manager_impl.cc:63] shutting down parent after drain

function-discovery

"Wed, 30 May 2018 06:58:15 UTC: github.com/solo-io/gloo/internal/function-discovery/detector/detector.go:64"	no more retries for "gloo-system-ingress-8080"
"Wed, 30 May 2018 06:58:15 UTC: github.com/solo-io/gloo/internal/function-discovery/detector/detector.go:64"	no more retries for "gloo-system-ingress-8443"
"Wed, 30 May 2018 06:58:15 UTC: github.com/solo-io/gloo/internal/function-discovery/detector/detector.go:64"	no more retries for "gloo-system-control-plane-8081"
"Wed, 30 May 2018 06:58:15 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:256"	updated upstream &v1.Upstream{
  Name:              "default-grpc-bookstore-8080",
  Type:              "kubernetes",
  ConnectionTimeout: 0,
  Spec:              &types.Struct{
    Fields: {
      "service_port": &types.Value{
        Kind: &types.Value_NumberValue{
          NumberValue: 8080.000000,
        },
      },
      "service_name": &types.Value{
        Kind: &types.Value_StringValue{
          StringValue: "grpc-bookstore",
        },
      },
      "service_namespace": &types.Value{
        Kind: &types.Value_StringValue{
          StringValue: "default",
        },
      },
    },
  },
  Functions:   []*v1.Function{},
  ServiceInfo: &v1.ServiceInfo{
    Type:       "gRPC",
    Properties: &types.Struct{
      Fields: {
        "descriptors_file_ref": &types.Value{
          Kind: &types.Value_StringValue{
            StringValue: "grpc-discovery:Bookstore.descriptors",
          },
        },
        "service_names": &types.Value{
          Kind: &types.Value_ListValue{
            ListValue: &types.ListValue{
              Values: []*types.Value{
                &types.Value{
                  Kind: &types.Value_StringValue{
                    StringValue: "Bookstore",
                  },
                },
              },
            },
          },
        },
      },
    },
  },
  Status: &v1.Status{
    State:  1,
    Reason: "",
  },
  Metadata: &v1.Metadata{
    ResourceVersion: "1518",
    Namespace:       "gloo-system",
    Annotations:     {
      "generated_by":                                     "kubernetes-upstream-discovery",
      "kubectl.kubernetes.io/last-applied-configuration": "{\"apiVersion\":\"v1\",\"kind\":\"Service\",\"metadata\":{\"annotations\":{},\"labels\":{\"sevice\":\"grpc-bookstore\"},\"name\":\"grpc-bookstore\",\"namespace\":\"default\"},\"spec\":{\"ports\":[{\"port\":8080,\"protocol\":\"TCP\"}],\"selector\":{\"app\":\"grpc-bookstore\"},\"type\":\"LoadBalancer\"}}\n",
      "gloo.solo.io/discovery-type":                      "grpc",
    },
  },
}
"Wed, 30 May 2018 06:58:15 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:98"	attempting update for "default-grpc-bookstore-8080"
"Wed, 30 May 2018 06:58:15 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:222"	attempting to apply update for upstream "default-grpc-bookstore-8080"
"Wed, 30 May 2018 06:58:16 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:98"	attempting update for "default-grpc-bookstore-8080"
"Wed, 30 May 2018 06:58:16 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:222"	attempting to apply update for upstream "default-grpc-bookstore-8080"
"Wed, 30 May 2018 07:15:44 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:112"	beginning update for []string{
  "gloo-system-control-plane-8081",
  "gloo-system-ingress-8080",
  "gloo-system-ingress-8443",
  "default-grpc-bookstore-8080",
} upstreams: 4
"Wed, 30 May 2018 07:15:44 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:98"	attempting update for "default-grpc-bookstore-8080"
"Wed, 30 May 2018 07:15:44 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:222"	attempting to apply update for upstream "default-grpc-bookstore-8080"
"Wed, 30 May 2018 07:15:44 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:98"	attempting update for "gloo-system-control-plane-8081"
"Wed, 30 May 2018 07:15:44 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:222"	attempting to apply update for upstream "gloo-system-control-plane-8081"
"Wed, 30 May 2018 07:15:44 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:112"	beginning update for []string{
  "gloo-system-control-plane-8081",
  "gloo-system-ingress-8080",
  "gloo-system-ingress-8443",
  "default-grpc-bookstore-8080",
} upstreams: 4
"Wed, 30 May 2018 07:15:44 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:98"	attempting update for "gloo-system-ingress-8080"
"Wed, 30 May 2018 07:15:44 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:222"	attempting to apply update for upstream "gloo-system-ingress-8080"
"Wed, 30 May 2018 07:15:44 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:112"	beginning update for []string{
  "gloo-system-control-plane-8081",
  "gloo-system-ingress-8080",
  "gloo-system-ingress-8443",
  "default-grpc-bookstore-8080",
} upstreams: 4
"Wed, 30 May 2018 07:15:44 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:98"	attempting update for "gloo-system-ingress-8443"
"Wed, 30 May 2018 07:15:44 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:222"	attempting to apply update for upstream "gloo-system-ingress-8443"
"Wed, 30 May 2018 07:15:44 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:112"	beginning update for []string{
  "gloo-system-control-plane-8081",
  "gloo-system-ingress-8080",
  "gloo-system-ingress-8443",
  "default-grpc-bookstore-8080",
} upstreams: 4
"Wed, 30 May 2018 07:15:44 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:112"	beginning update for []string{
  "gloo-system-control-plane-8081",
  "gloo-system-ingress-8080",
  "gloo-system-ingress-8443",
  "default-grpc-bookstore-8080",
} upstreams: 4
"Wed, 30 May 2018 07:15:44 UTC: github.com/solo-io/gloo/internal/function-discovery/detector/detector.go:64"	no more retries for "gloo-system-control-plane-8081"
"Wed, 30 May 2018 07:15:44 UTC: github.com/solo-io/gloo/internal/function-discovery/detector/detector.go:64"	no more retries for "gloo-system-ingress-8080"
"Wed, 30 May 2018 07:15:44 UTC: github.com/solo-io/gloo/internal/function-discovery/detector/detector.go:64"	no more retries for "gloo-system-ingress-8443"
"Wed, 30 May 2018 07:15:44 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:98"	attempting update for "default-grpc-bookstore-8080"
"Wed, 30 May 2018 07:15:44 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:222"	attempting to apply update for upstream "default-grpc-bookstore-8080"
"Wed, 30 May 2018 07:15:44 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:98"	attempting update for "gloo-system-control-plane-8081"
"Wed, 30 May 2018 07:15:44 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:222"	attempting to apply update for upstream "gloo-system-control-plane-8081"
"Wed, 30 May 2018 07:15:44 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:98"	attempting update for "gloo-system-ingress-8080"
"Wed, 30 May 2018 07:15:44 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:222"	attempting to apply update for upstream "gloo-system-ingress-8080"
"Wed, 30 May 2018 07:15:44 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:98"	attempting update for "gloo-system-ingress-8443"
"Wed, 30 May 2018 07:15:44 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:222"	attempting to apply update for upstream "gloo-system-ingress-8443"
"Wed, 30 May 2018 07:15:44 UTC: github.com/solo-io/gloo/internal/function-discovery/detector/detector.go:64"	no more retries for "gloo-system-control-plane-8081"
"Wed, 30 May 2018 07:15:45 UTC: github.com/solo-io/gloo/internal/function-discovery/detector/detector.go:64"	no more retries for "gloo-system-ingress-8080"
"Wed, 30 May 2018 07:15:45 UTC: github.com/solo-io/gloo/internal/function-discovery/detector/detector.go:64"	no more retries for "gloo-system-ingress-8443"
"Wed, 30 May 2018 07:15:45 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:98"	attempting update for "default-grpc-bookstore-8080"
"Wed, 30 May 2018 07:15:45 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:222"	attempting to apply update for upstream "default-grpc-bookstore-8080"
"Wed, 30 May 2018 07:15:46 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:98"	attempting update for "gloo-system-control-plane-8081"
"Wed, 30 May 2018 07:15:46 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:222"	attempting to apply update for upstream "gloo-system-control-plane-8081"
"Wed, 30 May 2018 07:15:46 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:98"	attempting update for "gloo-system-ingress-8080"
"Wed, 30 May 2018 07:15:46 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:222"	attempting to apply update for upstream "gloo-system-ingress-8080"
"Wed, 30 May 2018 07:15:47 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:98"	attempting update for "gloo-system-ingress-8443"
"Wed, 30 May 2018 07:15:47 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:222"	attempting to apply update for upstream "gloo-system-ingress-8443"
"Wed, 30 May 2018 07:15:47 UTC: github.com/solo-io/gloo/internal/function-discovery/detector/detector.go:64"	no more retries for "gloo-system-control-plane-8081"
"Wed, 30 May 2018 07:15:47 UTC: github.com/solo-io/gloo/internal/function-discovery/detector/detector.go:64"	no more retries for "gloo-system-ingress-8080"
"Wed, 30 May 2018 07:15:47 UTC: github.com/solo-io/gloo/internal/function-discovery/detector/detector.go:64"	no more retries for "gloo-system-ingress-8443"
"Wed, 30 May 2018 07:15:48 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:98"	attempting update for "default-grpc-bookstore-8080"
"Wed, 30 May 2018 07:15:48 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:222"	attempting to apply update for upstream "default-grpc-bookstore-8080"
"Wed, 30 May 2018 07:15:48 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:98"	attempting update for "gloo-system-control-plane-8081"
"Wed, 30 May 2018 07:15:48 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:222"	attempting to apply update for upstream "gloo-system-control-plane-8081"
"Wed, 30 May 2018 07:15:49 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:98"	attempting update for "gloo-system-ingress-8080"
"Wed, 30 May 2018 07:15:49 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:222"	attempting to apply update for upstream "gloo-system-ingress-8080"
"Wed, 30 May 2018 07:15:49 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:98"	attempting update for "gloo-system-ingress-8443"
"Wed, 30 May 2018 07:15:49 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:222"	attempting to apply update for upstream "gloo-system-ingress-8443"
"Wed, 30 May 2018 07:15:49 UTC: github.com/solo-io/gloo/internal/function-discovery/detector/detector.go:64"	no more retries for "gloo-system-control-plane-8081"
"Wed, 30 May 2018 07:15:50 UTC: github.com/solo-io/gloo/internal/function-discovery/detector/detector.go:64"	no more retries for "gloo-system-ingress-8080"
"Wed, 30 May 2018 07:15:50 UTC: github.com/solo-io/gloo/internal/function-discovery/detector/detector.go:64"	no more retries for "gloo-system-ingress-8443"
"Wed, 30 May 2018 07:15:50 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:98"	attempting update for "default-grpc-bookstore-8080"
"Wed, 30 May 2018 07:15:50 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:222"	attempting to apply update for upstream "default-grpc-bookstore-8080"
"Wed, 30 May 2018 07:15:51 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:98"	attempting update for "gloo-system-control-plane-8081"
"Wed, 30 May 2018 07:15:51 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:222"	attempting to apply update for upstream "gloo-system-control-plane-8081"
"Wed, 30 May 2018 07:15:51 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:98"	attempting update for "gloo-system-ingress-8080"
"Wed, 30 May 2018 07:15:51 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:222"	attempting to apply update for upstream "gloo-system-ingress-8080"
"Wed, 30 May 2018 07:15:51 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:98"	attempting update for "gloo-system-ingress-8443"
"Wed, 30 May 2018 07:15:51 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:222"	attempting to apply update for upstream "gloo-system-ingress-8443"
"Wed, 30 May 2018 07:15:52 UTC: github.com/solo-io/gloo/internal/function-discovery/detector/detector.go:64"	no more retries for "gloo-system-control-plane-8081"
"Wed, 30 May 2018 07:15:52 UTC: github.com/solo-io/gloo/internal/function-discovery/detector/detector.go:64"	no more retries for "gloo-system-ingress-8080"
"Wed, 30 May 2018 07:15:52 UTC: github.com/solo-io/gloo/internal/function-discovery/detector/detector.go:64"	no more retries for "gloo-system-ingress-8443"

Upstream Discovery

ERROR: logging before flag.Parse: W0530 06:46:11.927839       1 client_config.go:529] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
"Wed, 30 May 2018 06:46:11 UTC: github.com/solo-io/gloo/pkg/storage/crd/crd_storage_client.go:103"	registering crd v1.crd{
  Plural:    "upstreams",
  Group:     "gloo.solo.io",
  Version:   "v1",
  Kind:      "Upstream",
  ShortName: "us",
}
"Wed, 30 May 2018 06:46:11 UTC: github.com/solo-io/gloo/pkg/storage/crd/crd_storage_client.go:103"	registering crd v1.crd{
  Plural:    "virtualservices",
  Group:     "gloo.solo.io",
  Version:   "v1",
  Kind:      "VirtualService",
  ShortName: "vs",
}
"Wed, 30 May 2018 06:46:11 UTC: github.com/solo-io/gloo/pkg/storage/crd/crd_storage_client.go:103"	registering crd v1.crd{
  Plural:    "virtualmeshes",
  Group:     "gloo.solo.io",
  Version:   "v1",
  Kind:      "VirtualMesh",
  ShortName: "vm",
}
2018/05/30 06:46:11 starting kubernetes service discovery
"Wed, 30 May 2018 06:46:11 UTC: github.com/solo-io/gloo/vendor/github.com/solo-io/kubecontroller/controller.go:113"	Starting "kube-upstream-discovery" controller
"Wed, 30 May 2018 06:46:11 UTC: github.com/solo-io/gloo/vendor/github.com/solo-io/kubecontroller/controller.go:116"	Waiting for informer caches to sync
"Wed, 30 May 2018 06:46:12 UTC: github.com/solo-io/gloo/vendor/github.com/solo-io/kubecontroller/controller.go:121"	Starting workers
"Wed, 30 May 2018 06:46:12 UTC: github.com/solo-io/gloo/vendor/github.com/solo-io/kubecontroller/controller.go:127"	Started workers
"Wed, 30 May 2018 06:46:12 UTC: github.com/solo-io/gloo/pkg/config/desired_state.go:101"	updating upstream "gloo-system-control-plane-8081"
"Wed, 30 May 2018 06:46:12 UTC: github.com/solo-io/gloo/pkg/config/desired_state.go:101"	updating upstream "gloo-system-control-plane-8081"
"Wed, 30 May 2018 06:47:11 UTC: github.com/solo-io/gloo/pkg/config/desired_state.go:101"	updating upstream "gloo-system-control-plane-8081"
"Wed, 30 May 2018 06:47:11 UTC: github.com/solo-io/gloo/pkg/config/desired_state.go:101"	updating upstream "gloo-system-ingress-8080"
"Wed, 30 May 2018 06:47:12 UTC: github.com/solo-io/gloo/pkg/config/desired_state.go:101"	updating upstream "gloo-system-ingress-8443"
"Wed, 30 May 2018 06:48:11 UTC: github.com/solo-io/gloo/pkg/config/desired_state.go:101"	updating upstream "gloo-system-control-plane-8081"
"Wed, 30 May 2018 06:48:11 UTC: github.com/solo-io/gloo/pkg/config/desired_state.go:101"	updating upstream "gloo-system-ingress-8080"
"Wed, 30 May 2018 06:48:11 UTC: github.com/solo-io/gloo/pkg/config/desired_state.go:101"	updating upstream "gloo-system-ingress-8443"
"Wed, 30 May 2018 06:49:11 UTC: github.com/solo-io/gloo/pkg/config/desired_state.go:101"	updating upstream "gloo-system-control-plane-8081"
"Wed, 30 May 2018 06:49:11 UTC: github.com/solo-io/gloo/pkg/config/desired_state.go:101"	updating upstream "gloo-system-ingress-8080"
"Wed, 30 May 2018 06:49:11 UTC: github.com/solo-io/gloo/pkg/config/desired_state.go:101"	updating upstream "gloo-system-ingress-8443"
"Wed, 30 May 2018 06:50:11 UTC: github.com/solo-io/gloo/pkg/config/desired_state.go:101"	updating upstream "gloo-system-control-plane-8081"
"Wed, 30 May 2018 06:50:12 UTC: github.com/solo-io/gloo/pkg/config/desired_state.go:101"	updating upstream "gloo-system-ingress-8080"
"Wed, 30 May 2018 06:50:12 UTC: github.com/solo-io/gloo/pkg/config/desired_state.go:101"	updating upstream "gloo-system-ingress-8443"
"Wed, 30 May 2018 06:51:11 UTC: github.com/solo-io/gloo/pkg/config/desired_state.go:101"	updating upstream "gloo-system-control-plane-8081"
"Wed, 30 May 2018 06:51:12 UTC: github.com/solo-io/gloo/pkg/config/desired_state.go:101"	updating upstream "gloo-system-ingress-8080"
"Wed, 30 May 2018 06:51:12 UTC: github.com/solo-io/gloo/pkg/config/desired_state.go:101"	updating upstream "gloo-system-ingress-8443"
"Wed, 30 May 2018 06:52:11 UTC: github.com/solo-io/gloo/pkg/config/desired_state.go:101"	updating upstream "gloo-system-control-plane-8081"
"Wed, 30 May 2018 06:52:12 UTC: github.com/solo-io/gloo/pkg/config/desired_state.go:101"	updating upstream "gloo-system-ingress-8080"
"Wed, 30 May 2018 06:52:12 UTC: github.com/solo-io/gloo/pkg/config/desired_state.go:101"	updating upstream "gloo-system-ingress-8443"
"Wed, 30 May 2018 06:53:11 UTC: github.com/solo-io/gloo/pkg/config/desired_state.go:101"	updating upstream "gloo-system-control-plane-8081"
"Wed, 30 May 2018 06:53:11 UTC: github.com/solo-io/gloo/pkg/config/desired_state.go:101"	updating upstream "gloo-system-ingress-8080"
"Wed, 30 May 2018 06:53:12 UTC: github.com/solo-io/gloo/pkg/config/desired_state.go:101"	updating upstream "gloo-system-ingress-8443"
"Wed, 30 May 2018 06:54:11 UTC: github.com/solo-io/gloo/pkg/config/desired_state.go:101"	updating upstream "gloo-system-control-plane-8081"
"Wed, 30 May 2018 06:54:11 UTC: github.com/solo-io/gloo/pkg/config/desired_state.go:101"	updating upstream "gloo-system-ingress-8080"
"Wed, 30 May 2018 06:54:11 UTC: github.com/solo-io/gloo/pkg/config/desired_state.go:101"	updating upstream "gloo-system-ingress-8443"
"Wed, 30 May 2018 06:55:11 UTC: github.com/solo-io/gloo/pkg/config/desired_state.go:101"	updating upstream "gloo-system-control-plane-8081"
"Wed, 30 May 2018 06:55:11 UTC: github.com/solo-io/gloo/pkg/config/desired_state.go:101"	updating upstream "gloo-system-ingress-8080"
"Wed, 30 May 2018 06:55:12 UTC: github.com/solo-io/gloo/pkg/config/desired_state.go:101"	updating upstream "gloo-system-ingress-8443"
"Wed, 30 May 2018 06:56:11 UTC: github.com/solo-io/gloo/pkg/config/desired_state.go:101"	updating upstream "gloo-system-control-plane-8081"
"Wed, 30 May 2018 06:56:11 UTC: github.com/solo-io/gloo/pkg/config/desired_state.go:101"	updating upstream "gloo-system-ingress-8080"
"Wed, 30 May 2018 06:56:11 UTC: github.com/solo-io/gloo/pkg/config/desired_state.go:101"	updating upstream "gloo-system-ingress-8443"
"Wed, 30 May 2018 06:57:11 UTC: github.com/solo-io/gloo/pkg/config/desired_state.go:101"	updating upstream "gloo-system-control-plane-8081"
@ilackarms
Copy link
Member

are you able to ping 10.0.2.15? that looks like an internal NAT ip that virtualbox assigns; it might be unreachable from your host (unless you ssh inside your minikube vm and run commands from there)

@jeanlucmongrain
Copy link
Author

ah that is right, the pod address is not reachable from the host.

I thought that minikube was taking care of such as simple use case

@jeanlucmongrain
Copy link
Author

jeanlucmongrain commented May 30, 2018

a simple ip route add 10.0.0.0/8 via $IP_VM did the trick.

$ ping 10.0.2.15
PING 10.0.2.15 (10.0.2.15) 56(84) bytes of data.
64 bytes from 10.0.2.15: icmp_seq=1 ttl=64 time=0.451 ms
^C
--- 10.0.2.15 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.451/0.451/0.451/0.000 ms

try to connect to it:

$ curl --output - $GRPC_URL
curl: (56) Recv failure: Connection reset by peer

$ telnet 10.0.2.15 32688
Trying 10.0.2.15...
Connected to 10.0.2.15.
Escape character is '^]'.
�dsfafdas
dsf
^C
^]
telnet> quit
Connection closed.

$ curl --output - $GRPC_URL/bookstore.Bookstore/ListShelves
curl: (56) Recv failure: Connection reset by peer

I don't see anything different in the logs for function-discovery

"Wed, 30 May 2018 14:38:39 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:98"	attempting update for "gloo-system-ingress-8443"
"Wed, 30 May 2018 14:38:39 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:222"	attempting to apply update for upstream "gloo-system-ingress-8443"
"Wed, 30 May 2018 14:38:39 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:98"	attempting update for "gloo-system-ingress-8080"
"Wed, 30 May 2018 14:38:39 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:222"	attempting to apply update for upstream "gloo-system-ingress-8080"
"Wed, 30 May 2018 14:38:39 UTC: github.com/solo-io/gloo/internal/function-discovery/detector/detector.go:64"	no more retries for "gloo-system-ingress-8443"
"Wed, 30 May 2018 14:38:39 UTC: github.com/solo-io/gloo/internal/function-discovery/detector/detector.go:64"	no more retries for "gloo-system-ingress-8080"
"Wed, 30 May 2018 14:38:39 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:98"	attempting update for "gloo-system-ingress-8443"
"Wed, 30 May 2018 14:38:39 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:222"	attempting to apply update for upstream "gloo-system-ingress-8443"
"Wed, 30 May 2018 14:38:39 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:98"	attempting update for "gloo-system-ingress-8080"
"Wed, 30 May 2018 14:38:39 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:222"	attempting to apply update for upstream "gloo-system-ingress-8080"
"Wed, 30 May 2018 14:38:39 UTC: github.com/solo-io/gloo/internal/function-discovery/detector/detector.go:64"	no more retries for "gloo-system-ingress-8080"
"Wed, 30 May 2018 14:38:39 UTC: github.com/solo-io/gloo/internal/function-discovery/detector/detector.go:64"	no more retries for "gloo-system-ingress-8443"
"Wed, 30 May 2018 14:38:39 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:98"	attempting update for "gloo-system-ingress-8080"
"Wed, 30 May 2018 14:38:39 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:222"	attempting to apply update for upstream "gloo-system-ingress-8080"
"Wed, 30 May 2018 14:38:39 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:98"	attempting update for "gloo-system-ingress-8443"
"Wed, 30 May 2018 14:38:39 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:222"	attempting to apply update for upstream "gloo-system-ingress-8443"
"Wed, 30 May 2018 14:38:39 UTC: github.com/solo-io/gloo/internal/function-discovery/detector/detector.go:64"	no more retries for "gloo-system-ingress-8080"
"Wed, 30 May 2018 14:38:40 UTC: github.com/solo-io/gloo/internal/function-discovery/detector/detector.go:64"	no more retries for "gloo-system-ingress-8443"
"Wed, 30 May 2018 14:38:40 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:98"	attempting update for "gloo-system-ingress-8080"
"Wed, 30 May 2018 14:38:40 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:222"	attempting to apply update for upstream "gloo-system-ingress-8080"
"Wed, 30 May 2018 14:38:40 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:98"	attempting update for "gloo-system-ingress-8443"
"Wed, 30 May 2018 14:38:40 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:222"	attempting to apply update for upstream "gloo-system-ingress-8443"
"Wed, 30 May 2018 14:38:41 UTC: github.com/solo-io/gloo/internal/function-discovery/detector/detector.go:64"	no more retries for "gloo-system-ingress-8080"
"Wed, 30 May 2018 14:38:41 UTC: github.com/solo-io/gloo/internal/function-discovery/detector/detector.go:64"	no more retries for "gloo-system-ingress-8443"
"Wed, 30 May 2018 14:38:41 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:98"	attempting update for "gloo-system-ingress-8080"
"Wed, 30 May 2018 14:38:41 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:222"	attempting to apply update for upstream "gloo-system-ingress-8080"
"Wed, 30 May 2018 14:38:42 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:98"	attempting update for "gloo-system-ingress-8443"
"Wed, 30 May 2018 14:38:42 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:222"	attempting to apply update for upstream "gloo-system-ingress-8443"
"Wed, 30 May 2018 14:38:42 UTC: github.com/solo-io/gloo/internal/function-discovery/detector/detector.go:64"	no more retries for "gloo-system-ingress-8080"
"Wed, 30 May 2018 14:38:42 UTC: github.com/solo-io/gloo/internal/function-discovery/detector/detector.go:64"	no more retries for "gloo-system-ingress-8443"
"Wed, 30 May 2018 14:38:43 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:98"	attempting update for "gloo-system-ingress-8080"
"Wed, 30 May 2018 14:38:43 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:222"	attempting to apply update for upstream "gloo-system-ingress-8080"
"Wed, 30 May 2018 14:38:43 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:98"	attempting update for "gloo-system-ingress-8443"
"Wed, 30 May 2018 14:38:43 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:222"	attempting to apply update for upstream "gloo-system-ingress-8443"
"Wed, 30 May 2018 14:38:43 UTC: github.com/solo-io/gloo/internal/function-discovery/detector/detector.go:64"	no more retries for "gloo-system-ingress-8080"
"Wed, 30 May 2018 14:38:43 UTC: github.com/solo-io/gloo/internal/function-discovery/detector/detector.go:64"	no more retries for "gloo-system-ingress-8443"
"Wed, 30 May 2018 14:38:44 UTC: github.com/solo-io/gloo/internal/function-discovery/nats-streaming/discover_nats.go:34"	attempting to detect NATS for "gloo-system-control-plane-8081"
WARNING: "Wed, 30 May 2018 14:38:46 UTC: github.com/solo-io/gloo/cmd/function-discovery/main.go:43"	discovery error: &errors.withStack{
  error: &errors.withMessage{
    cause: &errors.withStack{
      error: &errors.withMessage{
        cause: &errors.fundamental{
          msg:   "service type detection failed for gloo-system-control-plane-8081: 5 errors occurred:\n\n* not an openfaas upstream\n* not a fission upstream\n* listing services. are you sure control-plane.gloo-system.svc.cluster.local:8081 implements reflection?: rpc error: code = Unimplemented desc = unknown service grpc.reflection.v1alpha.ServerReflection\n* service at control-plane.gloo-system.svc.cluster.local:8081 does not implement swagger at a known endpoint, or was unreachable: 5 errors occurred:\n\n* could not perform HTTP GET on resolved addr: control-plane.gloo-system.svc.cluster.local:8081: Get http://control-plane.gloo-system.svc.cluster.local:8081/swagger.json: net/http: HTTP/1.x transport connection broken: malformed HTTP response \"\\x00\\x00\\x00\\x04\\x00\\x00\\x00\\x00\\x00\"\n* could not perform HTTP GET on resolved addr: control-plane.gloo-system.svc.cluster.local:8081: Get http://control-plane.gloo-system.svc.cluster.local:8081/swagger/docs/v1: net/http: HTTP/1.x transport connection broken: malformed HTTP response \"\\x00\\x00\\x00\\x04\\x00\\x00\\x00\\x00\\x00\"\n* could not perform HTTP GET on resolved addr: control-plane.gloo-system.svc.cluster.local:8081: Get http://control-plane.gloo-system.svc.cluster.local:8081/swagger/docs/v2: net/http: HTTP/1.x transport connection broken: malformed HTTP response \"\\x00\\x00\\x00\\x04\\x00\\x00\\x00\\x00\\x00\"\n* could not perform HTTP GET on resolved addr: control-plane.gloo-system.svc.cluster.local:8081: Get http://control-plane.gloo-system.svc.cluster.local:8081/v1/swagger: net/http: HTTP/1.x transport connection broken: malformed HTTP response \"\\x00\\x00\\x00\\x04\\x00\\x00\\x00\\x00\\x00\"\n* could not perform HTTP GET on resolved addr: control-plane.gloo-system.svc.cluster.local:8081: Get http://control-plane.gloo-system.svc.cluster.local:8081/v2/swagger: net/http: HTTP/1.x transport connection broken: malformed HTTP response \"\\x00\\x00\\x00\\x04\\x00\\x00\\x00\\x00\\x00\"\n* failed to connect to nats-streaming cluster: read tcp 172.17.0.5:36154->10.111.199.192:8081: i/o timeout",
          stack: &errors.stack{
            0x11447bf,
            0x13d3f4a,
            0x141f9ce,
            0x141fe00,
            0x455ac1,
          },
        },
        msg: "failed to discover whether gloo-system-control-plane-8081 is a functional upstream",
      },
      stack: &errors.stack{
        0x13d4019,
        0x141f9ce,
        0x141fe00,
        0x455ac1,
      },
    },
    msg: "updating upstream gloo-system-control-plane-8081",
  },
  stack: &errors.stack{
    0x141fbee,
    0x141fe00,
    0x455ac1,
  },
}
"Wed, 30 May 2018 14:38:46 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:98"	attempting update for "gloo-system-control-plane-8081"
"Wed, 30 May 2018 14:38:46 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:222"	attempting to apply update for upstream "gloo-system-control-plane-8081"
"Wed, 30 May 2018 14:38:46 UTC: github.com/solo-io/gloo/internal/function-discovery/detector/detector.go:64"	no more retries for "gloo-system-control-plane-8081"
"Wed, 30 May 2018 14:38:46 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:98"	attempting update for "gloo-system-control-plane-8081"
"Wed, 30 May 2018 14:38:46 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:222"	attempting to apply update for upstream "gloo-system-control-plane-8081"
"Wed, 30 May 2018 14:38:46 UTC: github.com/solo-io/gloo/internal/function-discovery/detector/detector.go:64"	no more retries for "gloo-system-control-plane-8081"
"Wed, 30 May 2018 14:38:46 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:98"	attempting update for "gloo-system-control-plane-8081"
"Wed, 30 May 2018 14:38:46 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:222"	attempting to apply update for upstream "gloo-system-control-plane-8081"
"Wed, 30 May 2018 14:38:46 UTC: github.com/solo-io/gloo/internal/function-discovery/detector/detector.go:64"	no more retries for "gloo-system-control-plane-8081"
"Wed, 30 May 2018 14:38:46 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:98"	attempting update for "gloo-system-control-plane-8081"
"Wed, 30 May 2018 14:38:46 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:222"	attempting to apply update for upstream "gloo-system-control-plane-8081"
"Wed, 30 May 2018 14:38:47 UTC: github.com/solo-io/gloo/internal/function-discovery/detector/detector.go:64"	no more retries for "gloo-system-control-plane-8081"
"Wed, 30 May 2018 14:38:47 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:98"	attempting update for "gloo-system-control-plane-8081"
"Wed, 30 May 2018 14:38:47 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:222"	attempting to apply update for upstream "gloo-system-control-plane-8081"
"Wed, 30 May 2018 14:38:47 UTC: github.com/solo-io/gloo/internal/function-discovery/detector/detector.go:64"	no more retries for "gloo-system-control-plane-8081"
"Wed, 30 May 2018 14:38:48 UTC: github.com/solo-io/gloo/internal/function-discovery/eventloop/event_loop.go:98"	attempting update for "gloo-system-control-plane-8081"
"Wed, 30 May 2018 14:38:48 UTC: github.com/solo-io/gloo/internal/function-discovery/updater/updater.go:222"	attempting to apply update for upstream "gloo-system-control-plane-8081"
"Wed, 30 May 2018 14:38:48 UTC: github.com/solo-io/gloo/internal/function-discovery/detector/detector.go:64"	no more retries for "gloo-system-control-plane-8081"

@ilackarms
Copy link
Member

can you show:

glooctl upstream get
glooctl virtualservice get

@jeanlucmongrain
Copy link
Author

$ glooctl upstream get
+--------------------------------+------------+----------+----------+
|              NAME              |    TYPE    |  STATUS  | FUNCTION |
+--------------------------------+------------+----------+----------+
| default-grpc-bookstore-8080    | gRPC       | Accepted |          |
| gloo-system-control-plane-8081 | kubernetes | Accepted |          |
| gloo-system-ingress-8080       | kubernetes | Accepted |          |
| gloo-system-ingress-8443       | kubernetes | Accepted |          |
+--------------------------------+------------+----------+----------+
$ glooctl virtualservice get
+-----------------+--------+-----+--------+---------+------+------+--------+----------+----------+-----------+
| VIRTUAL SERVICE | DOMAIN | SSL | STATUS | MATCHER | TYPE | VERB | HEADER | UPSTREAM | FUNCTION | EXTENSION |
+-----------------+--------+-----+--------+---------+------+------+--------+----------+----------+-----------+
+-----------------+--------+-----+--------+---------+------+------+--------+----------+----------+-----------+

@ilackarms
Copy link
Member

you have no virtualservices defined. no virtualservice means no routes, which means Envoy isn't even listening (you'll get connection refused).

have you tried glooctl route create yet?

@ilackarms
Copy link
Member

btw if you pull a new image of gloo (redo the kubectl apply -f install/kube/install.yaml

@jeanlucmongrain
Copy link
Author

ok I think I found where I screw up
thanks

yuval-k added a commit that referenced this issue Nov 3, 2023
* i think this was a bad merge

* fix test
jenshu pushed a commit that referenced this issue Jan 18, 2024
* i think this was a bad merge

* fix test
jenshu added a commit that referenced this issue Feb 1, 2024
* Sketch controller (#45)

* controller sketch

* controllers

* helper functions, indexers

* maincompile

* main

* wip

* query

* gen-cli

* query secrets

* example-code

* git mv edge2/ gateway2

* cleanup

* cleanup2

* cleanup3

* Improve CI for gateway-main (#46)

* rename .github to disable github actions

* change cloudbuild.yaml to prevent build-bot running

* change query to have object in the args (#47)

* change query to have object in the args

To confirm that we use the right "from" field, ask for the whole object in the query args, and not just the ns

* use obj and infer gk

* fix comment

comments are bad

* singular

* some tests for query (#49)

* some tests for query

* more tests and pr comments

* more tests and fixes

* mroe test

* build + docker +helm + comformance (#51)

* ability to deploy to kind so we can run conformance

not yet working

* helming around, accepting status for gw class

* set observed generation on condition

* don't create gw for conformance test

* minor changes

* cleanup

* separate cp and dp

* cleanup

* allow more than one ports

* pr comments

* controlPlane.enabled

* fix makefile

* Gw main/ listener translation (#61)

* solo gw stubs

* wip: translator

* wip: http gwv2 translation

* wip: listener tx complete, ready for vhost + validation

* merge http filter chain

* finish core listener tx; stub vhost tx

* remove unused

* Auto provision gateway resources (#62)

* initial code to create GW

* auto provision by default
add ports to deployer

* fixes and some PR comments

* fix event filtering

* write addresses to gw; need to add tests

* only reconcile gw when generation changes

* support provisioning with different releases

* try approach 2 for statues

* env test for status setting

* pr comments

* update to v1 (#63)

* fix gw route query to make sure the http routes match (#64)

* fix gw route query to make sure the http routes match

* add and pass ci

* more ci

* make in g2 dir

* naming

* naming

* xds syncer (#60)

Co-authored-by: Eitan Yarmush <eitan.yarmush@solo.io>

* Initial validate (#67)

* solo gw stubs

* wip: translator

* wip: http gwv2 translation

* wip: listener tx complete, ready for vhost + validation

* merge http filter chain

* wip: initial impl of GW/Listener validation

* wip: working sketch of reporter; passing tests

* wip: begin testing for conditions

* simplify condition type on reporter

* add status checks to tests

* move listener validation back to listener package

* add gateway structure to report impl

* use ListenerStatus api type

* add support for allowedRoutes

* cleaup

* add tests for explicit allowedRoutes

* cleanup and reorg

* correctly track invalid listeners & routes

* more cleanup

* todo

---------

Co-authored-by: Scott Weiss <sdw35@cornell.edu>

* Gw main/translation httproute (#65)

* solo gw stubs

* wip: translator

* wip: http gwv2 translation

* wip: listener tx complete, ready for vhost + validation

* merge http filter chain

* finish core listener tx; stub vhost tx

* remove unused

* wip: route translator

* http route translator, fixes for http listener

* clean up todos

* Stub Out Endpoints and Upstream Discovery (#59)

* Add simple uds transslator

* goimports -w .

* format plugin

* update kube plugin to export a function that our new discvoery can rely on

* add stubs for translator for discovery

* add placeholder for legacy impl

* formatting

* simplify code

* add changelog

* query part 2 (#66)

* query part 2

* add parentref

* add conditions

* compile fix

* compile fix after merge

* simple secrets reconciler (#68)

* first pass route validation, handle nil hostnames on listeners (#70)

* create ListenerCondition type

* fix unnecessary guard on map access (go-static)

* add initial route validation

* handle empty/nil hostnames on listeners

* Gw main/translation yaml test (#72)

* testing and fixes for http translation

* get test passing

* nit

* fix import cycle, gofmt

* remove workaround

* add a server to tie everything together (#71)

* add a server to tie everything together

* more wiring

* enabled devmode

* de-pike

* Eitanya/single translation (#74)

* single translation spot

* compiles and runs

* fix compile issue in tests

* Eitanya/route sorting (#75)

* initial route-sorting

* tests are passing

* update golden

* swapped it all to use the httpRoute and idx as fallbacks

* also check NS/NAME

* oops

* move sortable to separate package

* initial sync status (#78)

* first drop working listener status

* handle gateway top-level conditions

* make xds work (#77)

* make xds work
main bug is there were 2 caches

* use envoy-gloo; use unprivileged port

* use rc2 for conformance to work

* pr comments

* tidy

* add no conflicts to healthy listeners (#79)

* add no conflicts to healthy listeners

* add resolvedrefs to healthy listeners

* add per-conditionType logic for missing conditions

* ssl termination (#76)

* ssl termination

* add ssl secret validation

* add ssl secret validation

* Eitanya/check v2 (#73)

* gateway checking is done

* compiles

* route status code

* with v1 changes

* a couple more changes

* remove ingress/knative

* also apply CRDs with glooctl

* pike

* install/uninstall work

* oops

* fix conditions for Gateway Class

* change back protocol on svc

* removed default gateway from the chart

* default to LB

* oops

* fix env tests with svc status hack due to LB

* naming

* use index func logic

* check for existence of the CRDs

* upped template to v1, changed logic to be more permissinve

* gatewayName

* simple version of port translation (#83)

* Sync route status (#82)

* init route report

* cleanup

* switch to per-parentRef reporting on routes

* report on route-parent relationship errors

* sync route status

* cleanup

* don't checkout gw repo for tests (#84)

* don't checkout gw repo for tests

* add conformance build tag; simplify kind

* fmt

* rename helm-chart (#80)

* rename helm-chart

* rename folder

* change dir name

* updated values/versions

* add labels so that glooctl version works

* make new commands more flexible

* file for CRDs, add programmed condition

* bad comment

* change version to alpha1 instead

* comments

* rename the gateway class name

* use standar

* use relative dir

* attached routes (#87)

* Fixes statuses temporarily (#88)

* re-add glooctl proxy commands

* various status fixes

* report tls errors on listeners (#86)

* report tls errors on listeners

* check specific errors on cert validation

* fix make (#89)

* more ci (#90)

* dont pre-check secret ref (#91)

* first translator plugin: header modifier (#85)

* implement plugin arch, w/ header modifier

* fix compile

* fix test

* redirect plugin

* add plugin

* add unit tests

* whoops

* rename compoenents to official naming scheme (#92)

* rename compoenents to official naming scheme

* fix unit tests

---------

Co-authored-by: Yuval Kohavi <yuval.kohavi@gmail.com>

* make conformance-GatewayWithAttachedRoutes passing (#93)

* make conformance-GatewayWithAttachedRoutes passing

* set status for listener with meta

* fix reason

* update to ghcr (#94)

* set gwNs for ML and reverse cert/key for TLS validation (#96)

* Revert "rename compoenents to official naming scheme (#92)" (#97)

This reverts commit b79b0e2.

* fix header order and match type (#102)

* support all hostnames (#98)

* i think this was a bad merge (#99)

* i think this was a bad merge

* fix test

* fix conformance-HTTPRouteInvalidNonExistentBackendRef (#100)

* listeners with more than one port are working now (#101)

* add unknown kind error if kind not in scheme (#103)

Co-authored-by: Lawrence G <lawrence.gadban@solo.io>

* gw semantics (#104)

* fix edge case (#106)

* fix empty backedRef with filters (#107)

* testing release on tag (#105)

* update release-1.0.0 (that includes kubernetes-sigs/gateway-api#2548) (#108)

* version fixes (#109)

* release glooctl

* oops

* Makefile

* no windows

* add https redirect hopefully

* url rewrite basic

* fix redirect and run conformnce

* add more conformance tests

* upload SHA

* use proper rewrite

* change regex

* works

* fix Makefile phony targets

* Gloo Gateway v2 readme (#8866)

* Gloo Gateway v2 readme

* added quickstart instructions

* added notes

* Update README.md

* some fixes

* Add files via upload

* Update README.md

* Update README.md

* fixed the ordering and added local HTTPBIN

* switched to local file

* Update README.md

---------

Co-authored-by: Eitan Yarmush <eitan.yarmush@solo.io>

* make README even simpler

* Logo and description update (#8872)

* Add files via upload

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* remove old flags from glooctl (#8871)

* Logo update (#8878)

* Add files via upload

* Add files via upload

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Revert to proxy ir (#8945)

* Adding quickstart for Helm (#8884)

* Adding quickstart for Helm

* Update README.md

* add logs

patch modified off PR (#8886) as we no longer want the native translation
updates.

Co-authored-by: Eitan Yarmush <eitan.yarmush@solo.io>
Co-authored-by: Lawrence Gadban <lawrence.gadban@solo.io>

* fix no match (#8879)

* Add Request Mirror Gateway API Filter (#8890)

Updated for Gloo Edge Proxy IR

---------

Co-authored-by: Yuval Kohavi <yuval.kohavi@gmail.com>

* add logs from previous PR #8892

* first drop of status cleanup (#8889)

* use types.NN for GatewayReports

* consolidate report creation

* reduce visibility of GatewayReport fields

* extract status building from syncer

* more narrowing viz, add Getters, first unit test for status building

* factor out condition boilerplate, more tests

* fix missed compile issue in translator tests

* add basic test for LastTransitionTime

* better testing

* cleanup

* Update projects/gateway2/reports/status.go

Co-authored-by: Eitan Yarmush <eitan.yarmush@solo.io>

* consolidate reporting components

---------

Co-authored-by: Eitan Yarmush <eitan.yarmush@solo.io>
Co-authored-by: Yuval Kohavi <yuval.kohavi@gmail.com>

* add label helper to helm templates

extracted from previous PR: make go test ./... work again (#8907)
above PR also fixed the CLI tests (e.g. glooctl check) to work with
GGv2. TODO has been added to add this back in later, but did not want to
take this on as the existing patches remove the v1 tests.

* refactor route status

cherry-pick of refactor route status (#8912)

* add missed conformance args from orig. PR (#8890)

* kick ci

---------

Co-authored-by: Nadine Spies <17709352+Nadine2016@users.noreply.github.com>
Co-authored-by: Yuval Kohavi <yuval.kohavi@gmail.com>
Co-authored-by: Eitan Yarmush <eitan.yarmush@solo.io>

* add RoutePlugins (#9021)

* wip: working RouteOptions as ExtensionRef

* add new format for extensionplugins

* extract extensionplugins to a discrete registry

* review feedback

* cleaup

* cleanup

* condense filter/plugin apply

* move routeoptions extension plugin to correct package structure

* cleanup

* introduce RoutePlugins with the impl for HTTPFilterPlugins

* remove extensionRef for now

* package structure re-org

* remove RouteOptions work for now

* localize httpfilterpluginregistry

* convert filterplugins to RoutePlugins

* generalize PluginRegistry (#9025)

* generalize PluginRegistry

* commentary

* add compile-time type assertions for plugins

* update name of RoutePlugin method

* use full type definition for base plugin interface

* track generation at report time (#8996)

* cleanup

* track generation at report time

* use common buildRoutesPerHost() for http & https

* enforce reports must be initialized for translated object

* handle missing reports gracefully

* commentary

* chore: reduce plugins visibility

* add RouteOption route plugin (#9040)

* add RouteOption route plugin

* cleanup

* commentary

* graceful & correct error handling

* reenable gh workflows

* add back some files

* remove changelogs

* revert changelogs

* remove changelogs for now

* add back cli stuff

* add back deleted stuff

* fix lint errors

* add back ingress

* docs gen

* v1 vs v2

* skip gateway2 for now

* fix?

* try reverting

* remove cl

* add back portredirect

* only set port redirect if not 0

* merge from main

* add translation mode

* more descriptive

* use new flag

* add field to test proxy

* remove new field from Proxy

* use label

* remove cl

* delete file

* add labels to test

* revert changelogs

* try again

* fix changelogs

* Adding changelog file to new location

* Deleting changelog file from old location

* codegen

* add back knative delete func

* change port_redirect to UInt32Value

* Adding changelog file to new location

* Deleting changelog file from old location

* some cleanup

* use context

* Update README.md

---------

Co-authored-by: Yuval Kohavi <yuval.kohavi@gmail.com>
Co-authored-by: Sam Heilbron <SamHeilbron@gmail.com>
Co-authored-by: ilackarms <sdw35@cornell.edu>
Co-authored-by: Eitan Yarmush <eitan.yarmush@solo.io>
Co-authored-by: Lawrence G <lawrence.gadban@solo.io>
Co-authored-by: Nadine Spies <17709352+Nadine2016@users.noreply.github.com>
Co-authored-by: soloio-bulldozer[bot] <48420018+soloio-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: changelog-bot <changelog-bot>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants