Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix visibility of pipeline metrics along with reserved/duplicate node name detection #648

Merged
merged 5 commits into from
Dec 14, 2020

Conversation

anupdhml
Copy link
Contributor

@anupdhml anupdhml commented Dec 11, 2020

Pull request

Fix generation of pipeline metrics (as reported in #647) and improve handing of cases when a pipeline node is created with the same name as an existing one.

Description

Ensured consistent id for metrics node when it's added to the pipeline graph (as "metrics"). Added tests to verify pipeline and ramp metrics as well.

Also throw error and terminate pipeline creation, if a user happens to use id metrics for one of their trickle operators (earlier, this was allowed and we would stop getting metrics too) -- this is solved as part of general handling described in #650.

For later, we should rename the events measurement to reflect that it's pipeline specific (or merge metrics in ramp_events under events.)

Related

Checklist

  • The RFC, if required, has been submitted and approved
  • Any user-facing impact of the changes is reflected in docs.tremor.rs
  • The code is tested
  • Use of unsafe code is reasoned about in a comment
  • Update CHANGELOG.md appropriately, recording any changes, bug fixes or other observable changes in behavior

Ramp metrics should be visible though

Signed-off-by: Anup Dhamala <anupdhml+git@gmail.com>
Signed-off-by: Anup Dhamala <anupdhml+git@gmail.com>
@coveralls
Copy link
Collaborator

coveralls commented Dec 11, 2020

Coverage Status

Coverage decreased (-0.05%) to 80.046% when pulling c8f7e7c on fix-pipeline-metrics into 0f5af59 on main.

Signed-off-by: Anup Dhamala <anupdhml+git@gmail.com>
@mfelsche
Copy link
Member

I think we should definitely throw an error if a user names a node in, out, err or metrics as these are reserved port, and thus pipeline operator names.

@anupdhml
Copy link
Contributor Author

Opened #650 to generalize error reporting when a node already exists with the given name.

… name

Fixes #650

Signed-off-by: Anup Dhamala <anupdhml+git@gmail.com>
@anupdhml anupdhml changed the title Fix visibility of pipeline metrics Fix visibility of pipeline metrics along with reserved/duplicate node name detection Dec 14, 2020
Copy link
Member

@mfelsche mfelsche left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM in general, I especially like the nice errors.
I just had a tiny question.

@anupdhml anupdhml merged commit b238ba6 into main Dec 14, 2020
@anupdhml anupdhml deleted the fix-pipeline-metrics branch December 14, 2020 18:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Monitoring : Pipeline metrics are not generated.
3 participants