Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Pipeline CR updates #68

Closed
6 tasks done
thedodd opened this issue Aug 17, 2021 · 0 comments · Fixed by #75
Closed
6 tasks done

Pipeline CR updates #68

thedodd opened this issue Aug 17, 2021 · 0 comments · Fixed by #75
Labels
A-crd Hadron K8s CRDs
Projects
Main

Comments

@thedodd
Copy link
Collaborator

thedodd commented Aug 17, 2021 •
edited
Loading

  • When a pipeline is created, as it is simply another consumer of the input stream, we need to be able to allow users to specify the pipeline's starting point. We should support all of the same options as a normal consumer.
  • Update pipeline controller logic to handle this update.
  • Update pipeline CRD.
  • update tracing to include base pipeline info (name &c).
  • use 3 byte encoding prefix for pipeline metadata.
  • remove unnecessary pipeline name in pipeline instance model.
@thedodd thedodd created this issue from a note in Main (In progress) Aug 17, 2021
@thedodd thedodd added A-crd Hadron K8s CRDs server labels Aug 17, 2021
thedodd added a commit that referenced this issue Aug 24, 2021
@thedodd
Update Pipeline CRD to support starting point config
aa6a77b 
This update allows pipelines to be created with a configuration which
controls the starting point of the source stream from which the pipeline
should begin processing.

A few efficiency improvements are added as well for encoding pipeline
metadata, storage of data models, and fixes an issue with recording of
active pipeline instances.

closes #68
thedodd added a commit that referenced this issue Aug 24, 2021
@thedodd
Update Pipeline CRD to support starting point config
7e748dc 
This update allows pipelines to be created with a configuration which
controls the starting point of the source stream from which the pipeline
should begin processing.

A few efficiency improvements are added as well for encoding pipeline
metadata, storage of data models, and fixes an issue with recording of
active pipeline instances.

closes #68
@thedodd thedodd mentioned this issue Aug 24, 2021
Merged
Main automation moved this from In progress to Done Aug 24, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
A-crd Hadron K8s CRDs
Projects
Main
Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Update Pipeline CRD to support starting point config hadron-project/hadron
1 participant
@thedodd