Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pass Elastic errors as events, instead of logging them #681

Closed
mfelsche opened this issue Jan 8, 2021 · 1 comment · Fixed by #715
Closed

Pass Elastic errors as events, instead of logging them #681

mfelsche opened this issue Jan 8, 2021 · 1 comment · Fixed by #715
Labels
enhancement New feature or request offramp Offramps
Milestone

Comments

@mfelsche
Copy link
Member

mfelsche commented Jan 8, 2021

Describe the problem you are trying to solve

In order to properly react to errors coming from requests sent via elastic sink, the overall errors and the bulk-element errors should be sent as error events to the err port of the sink. Currently bulk element errors are only logged at the ERROR level, which might quickly fill up disks. Also with error events with a proper format, users are going to be able to handle failures according to their business logic (retry policies while maintaining idempotency etc. etc.)

Describe the solution you'd like

Send all sink errors as Error events. This includes bulk element errors. These might be sent as separate events or as part of a single event which contains an array of all bulk errors.

Notes

If this is an implementation of an RFC provide a URL
to the RFC this enhancement implements.

If this is a major enhancement or contribution an RFC may be required. It is ok to submit an enhancement
first and our core team will assist with major contributions. In general, major contributions should be
discussed with the community before submission.

@mfelsche mfelsche added enhancement New feature or request offramp Offramps labels Jan 8, 2021
@mfelsche
Copy link
Member Author

We want to extend this change to make the elastic sink a "Linked Transport" in that it will emit success and error events for each bulk item coming back.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request offramp Offramps
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants