Skip to content
This repository has been archived by the owner on Dec 1, 2018. It is now read-only.

Add Riemann sink #379

Closed
wants to merge 1 commit into from
Closed

Add Riemann sink #379

wants to merge 1 commit into from

Conversation

jfoy
Copy link
Contributor

@jfoy jfoy commented Jun 28, 2015

Fixes #293

@cadvisorJenkinsBot
Copy link

Can one of the admins verify this patch?

@googlebot
Copy link

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project, in which case you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed, please reply here (e.g. I signed it!) and we'll verify. Thanks.


  • If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address. Check your existing CLA data and verify that your email is set on your git commits.
  • If you signed the CLA as a corporation, please let us know the company's name.

@jfoy
Copy link
Contributor Author

jfoy commented Jun 28, 2015

CCLA signed under Whitepages.

@jfoy
Copy link
Contributor Author

jfoy commented Jul 1, 2015

/cc @aphyr

@vishh
Copy link
Contributor

vishh commented Jul 1, 2015

cc @vmarmol

@jfoy
Copy link
Contributor Author

jfoy commented Jul 13, 2015

Seems I can't see the failing e2e test; is there a way to retrieve the result?

@vmarmol
Copy link
Contributor

vmarmol commented Jul 13, 2015

We're having issues with the e2e, I'd ignore it for now.

@jfoy
Copy link
Contributor Author

jfoy commented Jul 13, 2015

OK. What else do we need to review before this can be merged?

@@ -0,0 +1,22 @@
{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we place a symlink to an existing service file instead of creating a new one?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@vishh
Copy link
Contributor

vishh commented Aug 6, 2015

Overall LGTM. Sorry for the delay in reviewing this PR. This needs a rebase as well.

@vishh
Copy link
Contributor

vishh commented Aug 6, 2015

And do squash your commits, once you have gone through the comments.

@jfoy
Copy link
Contributor Author

jfoy commented Aug 10, 2015

Great -- I'll get this caught up.

@vishh
Copy link
Contributor

vishh commented Aug 31, 2015

@jfoy did you get a chance to work on this PR?

@jfoy
Copy link
Contributor Author

jfoy commented Sep 1, 2015

I will -- thanks for your patience.

@vishh
Copy link
Contributor

vishh commented Sep 14, 2015

@jfoy: Ping.

@jfoy
Copy link
Contributor Author

jfoy commented Sep 15, 2015

Not forgotten. We've been iterating on this internally, and I have a set of changes ready to go, which I'll push with the rebase and squash.

@jfoy
Copy link
Contributor Author

jfoy commented Sep 17, 2015

Done and squashed.

@jfoy
Copy link
Contributor Author

jfoy commented Sep 17, 2015

What's going on with our CCLA? We signed one from Whitepages, which seems to have vanished.

@jfoy
Copy link
Contributor Author

jfoy commented Sep 17, 2015

Fixing dependencies to unblock unit tests. (How embarrassing.)

@googlebot
Copy link

CLAs look good, thanks!

1 similar comment
@googlebot
Copy link

CLAs look good, thanks!


deps:
go get github.com/tools/godep
go get github.com/progrium/go-extpoints
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @mvdan

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should not be here since we use the vendored go-extpoints version as seen in https://github.com/kubernetes/heapster/blob/8e4a8ba63e75fe234bedb4171b35553d6ea18981/Godeps/Godeps.json#L6

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@@ -0,0 +1 @@
../influxdb/heapster-service.json
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@vishh
Copy link
Contributor

vishh commented Sep 18, 2015

@jfoy: This is amazing! Can't wait to try it out. Can you post some documentation on how users can use Reimann with heapster? Maybe some sample alert configs.

Other than documentation, everything else LGTM.

@@ -13,6 +13,8 @@ install:
- go get github.com/axw/gocov/gocov
- go get golang.org/x/tools/cmd/cover
- go get github.com/mattn/goveralls
- go get github.com/onsi/ginkgo
- go get github.com/onsi/gomega
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These should probably be vendored. If not, I'm thinking that the riemann tests may break at any point.

Didn't godep save ./... add them?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, my mistake. I hadn't run godeps save.

@jfoy
Copy link
Contributor Author

jfoy commented Sep 18, 2015

That was weird. Reverting to the prior state until I figure out what's going on with godep.

@mvdan
Copy link
Contributor

mvdan commented Sep 22, 2015

@jfoy I can take care of the godep stuff if you like - remove all the godep changes and I'll do a PR on top of yours.

@jfoy
Copy link
Contributor Author

jfoy commented Sep 22, 2015

@mvdan Awesome, thank you. I would like to understand where the variation comes from, though -- if I do a 'godep restore' followed by a 'godep save -t', why does the resulting Godeps/Godeps.json show so many deleted packages?

@mvdan
Copy link
Contributor

mvdan commented Sep 22, 2015

Running exactly those commands, the answer is simple - you have to give ./... as an argument to godep save. If you don't, it will assume ., thus saving only the dependencies for the main heapster package.

@jfoy
Copy link
Contributor Author

jfoy commented Sep 22, 2015

Here's what I see:

201345m0:heapster jfoy$ godep restore
201345m0:heapster jfoy$ git status
On branch riemann_sink
Your branch is up-to-date with 'jfoy/riemann_sink'.
nothing to commit, working directory clean
201345m0:heapster jfoy$ godep save -t ./... github.com/progrium/go-extpoints
201345m0:heapster jfoy$ git status
On branch riemann_sink
Your branch is up-to-date with 'jfoy/riemann_sink'.
Changes not staged for commit:
  (use "git add/rm <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

    modified:   Godeps/Godeps.json
    deleted:    Godeps/_workspace/src/code.google.com/p/go-uuid/uuid/LICENSE
    deleted:    Godeps/_workspace/src/code.google.com/p/go-uuid/uuid/dce.go
    deleted:    Godeps/_workspace/src/code.google.com/p/go-uuid/uuid/doc.go
    deleted:    Godeps/_workspace/src/code.google.com/p/go-uuid/uuid/hash.go
    deleted:    Godeps/_workspace/src/code.google.com/p/go-uuid/uuid/node.go
    deleted:    Godeps/_workspace/src/code.google.com/p/go-uuid/uuid/time.go
    deleted:    Godeps/_workspace/src/code.google.com/p/go-uuid/uuid/util.go
    deleted:    Godeps/_workspace/src/code.google.com/p/go-uuid/uuid/uuid.go
    deleted:    Godeps/_workspace/src/code.google.com/p/go-uuid/uuid/uuid_test.go
    deleted:    Godeps/_workspace/src/code.google.com/p/go-uuid/uuid/version1.go
    deleted:    Godeps/_workspace/src/code.google.com/p/go-uuid/uuid/version4.go
    deleted:    Godeps/_workspace/src/github.com/docker/docker/pkg/mount/flags.go
    deleted:    Godeps/_workspace/src/github.com/docker/docker/pkg/mount/flags_freebsd.go
    deleted:    Godeps/_workspace/src/github.com/docker/docker/pkg/mount/flags_linux.go
    deleted:    Godeps/_workspace/src/github.com/docker/docker/pkg/mount/flags_unsupported.go
    deleted:    Godeps/_workspace/src/github.com/docker/docker/pkg/mount/mount.go
    deleted:    Godeps/_workspace/src/github.com/docker/docker/pkg/mount/mount_test.go
    deleted:    Godeps/_workspace/src/github.com/docker/docker/pkg/mount/mounter_freebsd.go
    deleted:    Godeps/_workspace/src/github.com/docker/docker/pkg/mount/mounter_linux.go
    deleted:    Godeps/_workspace/src/github.com/docker/docker/pkg/mount/mounter_unsupported.go
    deleted:    Godeps/_workspace/src/github.com/docker/docker/pkg/mount/mountinfo.go
    deleted:    Godeps/_workspace/src/github.com/docker/docker/pkg/mount/mountinfo_freebsd.go
    deleted:    Godeps/_workspace/src/github.com/docker/docker/pkg/mount/mountinfo_linux.go
    deleted:    Godeps/_workspace/src/github.com/docker/docker/pkg/mount/mountinfo_linux_test.go
    deleted:    Godeps/_workspace/src/github.com/docker/docker/pkg/mount/mountinfo_unsupported.go
    deleted:    Godeps/_workspace/src/github.com/docker/docker/pkg/mount/sharedsubtree_linux.go
    deleted:    Godeps/_workspace/src/github.com/docker/docker/pkg/mount/sharedsubtree_linux_test.go
    deleted:    Godeps/_workspace/src/github.com/docker/docker/pkg/units/duration.go
    deleted:    Godeps/_workspace/src/github.com/docker/docker/pkg/units/duration_test.go
    deleted:    Godeps/_workspace/src/github.com/docker/docker/pkg/units/size.go
    deleted:    Godeps/_workspace/src/github.com/docker/docker/pkg/units/size_test.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/cgroups/cgroups.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/cgroups/cgroups_test.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/cgroups/cgroups_unsupported.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/cgroups/fs/apply_raw.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/cgroups/fs/blkio.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/cgroups/fs/blkio_test.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/cgroups/fs/cpu.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/cgroups/fs/cpu_test.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/cgroups/fs/cpuacct.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/cgroups/fs/cpuset.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/cgroups/fs/cpuset_test.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/cgroups/fs/devices.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/cgroups/fs/devices_test.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/cgroups/fs/freezer.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/cgroups/fs/freezer_test.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/cgroups/fs/fs_unsupported.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/cgroups/fs/hugetlb.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/cgroups/fs/hugetlb_test.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/cgroups/fs/memory.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/cgroups/fs/memory_test.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/cgroups/fs/net_cls.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/cgroups/fs/net_cls_test.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/cgroups/fs/net_prio.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/cgroups/fs/net_prio_test.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/cgroups/fs/perf_event.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/cgroups/fs/stats_util_test.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/cgroups/fs/util_test.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/cgroups/fs/utils.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/cgroups/fs/utils_test.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/cgroups/stats.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/cgroups/systemd/apply_nosystemd.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/cgroups/systemd/apply_systemd.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/cgroups/utils.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/configs/cgroup.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/configs/config.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/configs/config_test.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/configs/config_unix.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/configs/device.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/configs/device_defaults.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/configs/hugepage_limit.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/configs/interface_priority_map.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/configs/mount.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/configs/namespaces.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/configs/namespaces_syscall.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/configs/namespaces_syscall_unsupported.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/configs/namespaces_unix.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/configs/namespaces_windows.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/configs/network.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/configs/validate/config.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/system/linux.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/system/proc.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/system/setns_linux.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/system/syscall_linux_386.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/system/syscall_linux_64.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/system/syscall_linux_arm.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/system/sysconfig.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/system/sysconfig_notcgo.go
    deleted:    Godeps/_workspace/src/github.com/docker/libcontainer/system/xattrs_linux.go
    modified:   Godeps/_workspace/src/github.com/prometheus/client_golang/text/bench_test.go
    deleted:    Godeps/_workspace/src/k8s.io/kubernetes/pkg/apis/experimental/install/install.go
    deleted:    Godeps/_workspace/src/k8s.io/kubernetes/pkg/client/unversioned/clientcmd/api/helpers.go
    deleted:    Godeps/_workspace/src/k8s.io/kubernetes/pkg/client/unversioned/clientcmd/api/latest/latest.go
    deleted:    Godeps/_workspace/src/k8s.io/kubernetes/pkg/client/unversioned/clientcmd/api/register.go
    deleted:    Godeps/_workspace/src/k8s.io/kubernetes/pkg/client/unversioned/clientcmd/api/types.go
    deleted:    Godeps/_workspace/src/k8s.io/kubernetes/pkg/client/unversioned/clientcmd/api/v1/conversion.go
    deleted:    Godeps/_workspace/src/k8s.io/kubernetes/pkg/client/unversioned/clientcmd/api/v1/register.go
    deleted:    Godeps/_workspace/src/k8s.io/kubernetes/pkg/client/unversioned/clientcmd/api/v1/types.go
    deleted:    Godeps/_workspace/src/k8s.io/kubernetes/pkg/client/unversioned/clientcmd/auth_loaders.go
    deleted:    Godeps/_workspace/src/k8s.io/kubernetes/pkg/client/unversioned/clientcmd/client_config.go
    deleted:    Godeps/_workspace/src/k8s.io/kubernetes/pkg/client/unversioned/clientcmd/doc.go
    deleted:    Godeps/_workspace/src/k8s.io/kubernetes/pkg/client/unversioned/clientcmd/loader.go
    deleted:    Godeps/_workspace/src/k8s.io/kubernetes/pkg/client/unversioned/clientcmd/merged_client_builder.go
    deleted:    Godeps/_workspace/src/k8s.io/kubernetes/pkg/client/unversioned/clientcmd/overrides.go
    deleted:    Godeps/_workspace/src/k8s.io/kubernetes/pkg/client/unversioned/clientcmd/validation.go
    deleted:    Godeps/_workspace/src/k8s.io/kubernetes/pkg/probe/doc.go
    deleted:    Godeps/_workspace/src/k8s.io/kubernetes/pkg/probe/exec/exec.go
    deleted:    Godeps/_workspace/src/k8s.io/kubernetes/pkg/probe/http/http.go
    deleted:    Godeps/_workspace/src/k8s.io/kubernetes/pkg/probe/probe.go
    deleted:    Godeps/_workspace/src/k8s.io/kubernetes/pkg/probe/tcp/tcp.go

Untracked files:
  (use "git add <file>..." to include in what will be committed)

    Godeps/_workspace/src/github.com/onsi/

no changes added to commit (use "git add" and/or "git commit -a")
201345m0:heapster jfoy$

@jfoy
Copy link
Contributor Author

jfoy commented Sep 22, 2015

@vishh I'm pulling together some config samples.

Support a Riemann sink (derived from the influxdb sink).

Map Heapster events into Riemann:
* Map Heapster event structures to Riemann events.
* Coerce Int64 metrics to Int for transport to Riemann.
* Support user-provided Riemann tags and event state, with defaults.

Includes basic documentation and sample Riemann config.
@mvdan
Copy link
Contributor

mvdan commented Sep 23, 2015

@jfoy funnily enough, I get the same thing you get when running save on master. It obviously breaks:

sources/kube_factory.go:30:2: cannot find package "k8s.io/kubernetes/pkg/client/unversioned/clientcmd" in any of:

Some of the things it does, like removing go-uuid, are fine - but others aren't. The best I can come up with now is that godep must be broken. We might be doing something wrong here, but the problem is just too obvious.

@vishh
Copy link
Contributor

vishh commented Sep 23, 2015

@cadvisorJenkinsBot: test this please

@jfoy
Copy link
Contributor Author

jfoy commented Sep 24, 2015

I can't tell by looking at the output -- is this another artifact of the e2e pipeline?

@mvdan
Copy link
Contributor

mvdan commented Sep 24, 2015

@jfoy see #601 - could you rebase your PR and try his trick of removing Godeps before running save?

@jimmidyson
Copy link
Contributor

It may be enough to just rebase & do godep save ./... github.com/progrium/go-extpoints now that godeps are cleaned up a bit. I'd try that first.

@mvdan
Copy link
Contributor

mvdan commented Sep 26, 2015

Godep is working fine for me now from a clean GOPATH, without removing Godeps/:

GOPATH=/tmp/go
mkdir $GOPATH
godep restore
godep save -v ./... github.com/progrium/go-extpoints

@vishh
Copy link
Contributor

vishh commented Oct 8, 2015

@jfoy: Any updates on this PR? I can help address the godeps issue if you are busy.

@jfoy
Copy link
Contributor Author

jfoy commented Oct 29, 2015

I would love help addressing the godeps issue. I'm anxious to get this merged.

@vishh
Copy link
Contributor

vishh commented Nov 5, 2015

@jfoy: godeps can be a pain at times. I went ahead and fixed godeps on HEAD and also included your patches in the process. I will open a separate PR that will include your patches in this PR. Sorry for the trouble with godeps.

@piosz
Copy link
Contributor

piosz commented Nov 5, 2015

closing this in favor of #687

@piosz piosz closed this Nov 5, 2015
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants