Release v0.12.1

Fixes

• Fix misbehaving stdio connector when stdio is a pipe
• Fix Docker entrypoint script argument ordering
• Fix id encoding for discord connector

Release v0.12.0

New features

• Add full TLS support for elastic connector (including client certificate auth)
• Add several auth methods for elastic and HTTP connector (Basic Auth, Api-Key, Bearer Token, ...)
• Add support for specifying client certificates with tls config for http_client and tcp_client connectors.
• Add tremor new to create new template projects.
• Support elastic.raw_payload for update
• Add support for chunked requests and responses for the HTTP connectors
• Add the gbq connector for Google BigQuery
• Add support for tuple patterns inside of record patterns.
• Refactor visitors to seperate walker and visitor and visit all nodes.
• Add support for modular subqueries in Trickle
• Add check_topic_metadata configuration flag to kafka source to bypass topic metadata fetch
• port json! improvements to literal!
• print names of failing tests
• Add check_topic_metadata configuration flag to kafka source to bypass topic metadata fetch
• port json! improvements to literal!
• Add support for troy deployment language. The language adds the pipeline, connector, flow, links, connect, to, deploy and config reserved keywords. These are now reserved in the scripting and query languages and must now be escaped when they appear in event data. This is a breaking change in the query and script language dialects.
• Add support for FLOAT4 and FLOAT8 serialization/deserialization to postgres connectors
• Add std::path::try_default fn
• Experimental ARM support
• Add default => {...} and default "key" => "value" to patch
• Add zstd pre- and post-processors #1100
• Remove rental from Event #1031 #1037
• Put event raw payload into Arc to improve cloning perf
• Remove rental from the entire runtime #1037
• Restructure operators and select to avoid transmutation #1024
• Fix tremor-cli to use it's own binary when possible #1096
• Restructure functions to avoid transmutation #1028
• Update blackhole to print events / s #1129
• Improve soundness and documentation of SRS code.
• Add support for concatenating arrays #1113
• Allow gcp headers to be included in rest offramp with auth: gcp
• Add env source #1136
• Remove need for emit_emtpy_window by automatic eviction for groups where no window holds data
• Ensure all windows of a group are alligned and can not go out of sync
• Box apropriately in the rest sink
• replace macros with functions in gcp code
• Add -q flag and clarify -v flag for unit tests
• Add win::cardinality function
• Backpressure now allows lossless (no discard) circuit breaker behaviour #1119
• Allow functions, constants and expressions as roots for path lookups
• Add the unix-socket source
• Enabale automatic benchmakrs
• Add the csv codec
• Add the permissions option for setting the file mode for the unix-socket source
• Tests can be run without their suite. #1238
• Add the std::size module to convert sizes
• Add custom function calls to constant folding
• Integration test names are added as a tag so they can be run by name
• integration tests low log stdout/stderr for before and after

Breaking Changes

• the tremor api sub command in the cli has been removed.
• the - is no longer a valid part of identifiers.
• binaries now use _ to sepoerate type names as - is no longer a identifier.
• changed naming for record object to avoid keywords like select and merge. New names are record.extract and record.combine.
• command separators are now unified, both patch, match and for now use ; the same way the rest of the language does
• in all definitional statements args now specifies interface arguments that are overwritable in the correspanding create statement, while with specifies non-overwritable configuration in both define and create statements - this unifies the use of with and args between trickle and troy
• file connector no longer splits by lines - it now requires a preconnector
• define for both troy and trickle now follow the same principle of define <type> <alias> from <source>
• wal is no longer an operator but a connector
• for the elastic connector indexes have not to be set on the batch not the individual event so one batch can only be to a single index.
• metronome interval is now in nanoseconds (as all other timings)
• Most connectors require a specified codec now instead of using JSON as a default
• merge no longer treats null in the spec as a delete option but rather as a normal value
• Combine all compression and decompression pre/postprocessors.

Fixes

• Fix detection of *.troy files in entrypoint.sh causing duplicate configs to be loaded when using Kubernetes.
• Fix a one-off error in the bench connector leading to it producing one event too much
• Avoid acking events that failed while preprocessing or decoding using a codec.
• Remove acknowledging events when dropped or sent to a dead end (e.g. unconnected port) as this was causing confusing and unwanted behaviour with error events
• move EXIT connector to debug connectors to avoid shutting down tremor
• Fix bug in grok extractor that would never return good matches
• Properly terminate after tremor run ...
• Fix the bench connector to actually stop after the given amount of events when iters is configured.
• http headers to allow strings and arrays
• fix the use of args in the with part of a create insode of a flow.
• fix silent swallowing of unknown fields in connector definition.
• Make otel severity_number optional: #1248
• Don't allow duplicate stream names: #1212
• Fix memory safety issue when using merge or patch with state as target and reassigning the resulting value to state #1217
• Fix delayed event re-execution in case of errors in a branched pipeline
• Skip instead of fail EQC on out of repo PRs
• Ensure patch keys are strings to move runtime errors into the compiletime
• Fix issue with the token " being presented as a tick in errors
• Fix heredoc_start and heredoc_end showing up in error messages instead of """
• Fix some errors in otel pb <-> json translation
• Fix windowed queries emitting events with null metadata on tick
• Fix sorting for artefacts
• Fix issue where the test framework would generate reports without being asked for it #1072
• Remove the need for eviction_period for time based windows
• Remove dead code and unneeded allows in otel and gcp code
• Fix test and suite names not being printed
• Fix badly nested structure in unit tests
• Fix bug in the unit testing framework that would ignore all tags
• Fix the illogical structure of suites that required a doubly nested record
• Fix -v flag
• Fix issue with double counting of unit test stats
• Fix issue with wrong script snippets being shown for unit tests
• Fix argument order in test cases
• Fix GCS go-auth token refresh
• Fix create script syntax for aliased scripts with overridden params
• Add benchmark names to benchmark tags
• Kafka onramp: Remove failing metadata fetch in order to verify topic existance. Instead detect subscription errors and stop the onramp in that case.
• Unix offramp: Add the missing StartStream message
• tremor-script: Add more details about Unicode in the documentation of the string module
• Fix hdr and dds aggregation function losing events when aggregating > 8192 events
• Ensure merge can only happen on objects

Release v0.12.0-rc.9

Fixes

• Fix detection of *.troy files in entrypoint.sh causing duplicate configs to be loaded when using Kubernetes.
• Fix a one-off error in the bench connector leading to it producing one event too much
• Avoid acking events that failed while preprocessing or decoding using a codec.
• Remove acknowledging events when dropped or sent to a dead end (e.g. unconnected port) as this was causing confusing and unwanted behaviour with error events
• move EXIT connector to debug connectors to avoid shutting down tremor
• Fix bug in grok extractor that would never return good matches

New features

• Add full TLS support for elastic connector (including client certificate auth)
• Add several auth methods for elastic and HTTP connector (Basic Auth, Api-Key, Bearer Token, ...)
• Add support for specifying client certificates with tls config for http_client and tcp_client connectors.
• Add tremor new to create new template projects.

Release v0.12.0-rc.8

Fixes

• Properly terminate after tremor run ...
• Fix the bench connector to actually stop after the given amount of events when iters is configured.

New features

• Support elastic.raw_payload for update

Release v0.12.0-rc.7

Fixes

• Properly terminate after tremor run ...
• Fix the bench connector to actually stop after the given amount of events when iters is configured.

New features

• Support elastic.raw_payload for update

Release v0.12.0-rc.6

Fixes

• http headers to allow strings and arrays

Release v0.12.0-rc.5

New features

• Add support for chunked requests and responses for the HTTP connectors
• Add the gbq connector for Google BigQuery

Fixes

• fix the use of args in the with part of a create insode of a flow.
• fix silent swallowing of unknown fields in connector definition.

Breaking Changes

• the - is no longer a valid part of identifiers.
• binaries now use _ to sepoerate type names as - is no longer a identifier.

Release v0.12.0-rc.4

New features

• Add support for chunked requests and responses for the HTTP connectors
• Add the gbq connector for Google BigQuery

Fixes

• fix the use of args in the with part of a create insode of a flow.
• fix silent swallowing of unknown fields in connector definition.

Breaking Changes

• the - is no longer a valid part of identifiers.
• binaries now use _ to sepoerate type names as - is no longer a identifier.

Release v0.12.0-rc.2

New features

• Add support for chunked requests and responses for the HTTP connectors
• Add the gbq connector for Google BigQuery

Fixes

• fix the use of args in the with part of a create insode of a flow.
• fix silent swallowing of unknown fields in connector definition.

Breaking Changes

• the - is no longer a valid part of identifiers.
• binaries now use _ to sepoerate type names as - is no longer a identifier.

Release v0.12.0-rc.1

0.12.0-rc.1

New featues

• Add support for tuple patterns inside of record patterns.
• Refactor visitors to seperate walker and visitor and visit all nodes.
• Add support for modular subqueries in Trickle
• Add check_topic_metadata configuration flag to kafka source to bypass topic metadata fetch
• port json! improvements to literal!
• print names of failing tests
• Add check_topic_metadata configuration flag to kafka source to bypass topic metadata fetch
• port json! improvements to literal!
• Add support for troy deployment language. The language adds the pipeline, connector, flow, links, connect, to, deploy and config reserved keywords. These are now reserved in the scripting and query languages and must now be escaped when they appear in event data. This is a breaking change in the query and script language dialects.
• Add support for FLOAT4 and FLOAT8 serialization/deserialization to postgres connectors
• Add std::path::try_default fn
• Experimental ARM support
• Add default => {...} and default "key" => "value" to patch
• Add zstd pre- and post-processors #1100
• Remove rental from Event #1031 #1037
• Put event raw payload into Arc to improve cloning perf
• Remove rental from the entire runtime #1037
• Restructure operators and select to avoid transmutation #1024
• Fix tremor-cli to use it's own binary when possible #1096
• Restructure functions to avoid transmutation #1028
• Update blackhole to print events / s #1129
• Improve soundness and documentation of SRS code.
• Add support for concatenating arrays #1113
• Allow gcp headers to be included in rest offramp with auth: gcp
• Add env source #1136
• Remove need for emit_emtpy_window by automatic eviction for groups where no window holds data
• Ensure all windows of a group are alligned and can not go out of sync
• Box apropriately in the rest sink
• replace macros with functions in gcp code
• Add -q flag and clarify -v flag for unit tests
• Add win::cardinality function
• Backpressure now allows lossless (no discard) circuit breaker behaviour #1119
• Allow functions, constants and expressions as roots for path lookups
• Add the unix-socket source
• Enabale automatic benchmakrs
• Add the csv codec
• Add the permissions option for setting the file mode for the unix-socket source
• Tests can be run without their suite. #1238
• Add the std::size module to convert sizes
• Add custom function calls to constant folding
• Integration test names are added as a tag so they can be run by name
• integration tests low log stdout/stderr for before and after

Fixes

• Make otel severity_number optional: #1248
• Don't allow duplicate stream names: #1212
• Fix memory safety issue when using merge or patch with state as target and reassigning the resulting value to state #1217
• Fix delayed event re-execution in case of errors in a branched pipeline
• Skip instead of fail EQC on out of repo PRs
• Ensure patch keys are strings to move runtime errors into the compiletime
• Fix issue with the token " being presented as a tick in errors
• Fix heredoc_start and heredoc_end showing up in error messages instead of """
• Fix some errors in otel pb <-> json translation
• Fix windowed queries emitting events with null metadata on tick
• Fix sorting for artefacts
• Fix issue where the test framework would generate reports without being asked for it #1072
• Remove the need for eviction_period for time based windows
• Remove dead code and unneeded allows in otel and gcp code
• Fix test and suite names not being printed
• Fix badly nested structure in unit tests
• Fix bug in the unit testing framework that would ignore all tags
• Fix the illogical structure of suites that required a doubly nested record
• Fix -v flag
• Fix issue with double counting of unit test stats
• Fix issue with wrong script snippets being shown for unit tests
• Fix argument order in test cases
• Fix GCS go-auth token refresh
• Fix create script syntax for aliased scripts with overridden params
• Add benchmark names to benchmark tags
• Kafka onramp: Remove failing metadata fetch in order to verify topic existance. Instead detect subscription errors and stop the onramp in that case.
• Unix offramp: Add the missing StartStream message
• tremor-script: Add more details about Unicode in the documentation of the string module
• Fix hdr and dds aggregation function losing events when aggregating > 8192 events
• Ensure merge can only happen on objects

Breaking Changes

• changed naming for record object to avoid keywords like select and merge. New names are record.extract and record.combine.
• command separators are now unified, both patch, match and for now use ; the same way the rest of the language does
• in all definitional statements args now specifies interface arguments that are overwritable in the correspanding create statement, while with specifies non-overwritable configuration in both define and create statements - this unifies the use of with and args between trickle and troy
• file connector no longer splits by lines - it now requires a preconnector
• define for both troy and trickle now follow the same principle of define <type> <alias> from <source>
• wal is no longer an operator but a connector
• for the elastic connector indexes have not to be set on the batch not the individual event so one batch can only be to a single index.
• metronome interval is now in nanoseconds (as all other timings)
• Most connectors require a specified codec now instead of using JSON as a default
• merge no longer treats null in the spec as a delete option but rather as a normal value
• Combine all compression and decompression pre/postprocessors.