KV API

[thedodd]

Under the hood, Hadron is built upon a robust KV embedded database. Let's explore some use cases where some level of access to the KV store — outside of the normal streaming interface — may be good and desirable.

OptAttr Controls

There will be plenty of cases where users will want to directly query/mutate the data of a stream or pipeline. Generally this violates the intention behind the design of the system. However there are still cases where some amount of access might be nice without breaking these constraints.

• During development, or when a bug has been rolled to a production environment, producers may be emitting malformed data, and the consumers will never be able to reasonably be able to process the task.
• In such cases, the data may be malformed enough that rolling some new code to parse it would be unreasonable.
• How might a user get past this issue?

Perhaps a reasonable path forward is to allow for optattr updates on events.

• For the case above, the event could be updated to include some new attribute on the event.
• Consumers could be coded to skip / ack events with such attributes.
• Similar to a dead-letter system.

Explore this a bit more. Perhaps we need something like an "official" Hadron CloudEvents extension field for marking events as dead-letter.

[thedodd]

Closing in favor of #100