Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Time Vector Templates #638

Merged
merged 1 commit into from
Dec 6, 2022
Merged

Time Vector Templates #638

merged 1 commit into from
Dec 6, 2022

Conversation

thatzopoulos
Copy link
Contributor

@thatzopoulos thatzopoulos commented Nov 28, 2022

These changes allow formatted String output from a Time Vector series. The format_timevector() fn takes args for a formatted string and a Time Vector Series. The Tera crate is used to populate the formatted string with Time Vector values and render it for the user. format_timevector() has two built in templates that can be used: paired and plotly.

@thatzopoulos thatzopoulos force-pushed the th/time-vector-template-json branch 5 times, most recently from 79fba01 to 46aa1f6 Compare November 28, 2022 21:28
@thatzopoulos thatzopoulos marked this pull request as ready for review November 28, 2022 21:28
extension/src/time_vector.rs Outdated Show resolved Hide resolved
extension/src/time_vector.rs Outdated Show resolved Hide resolved
extension/src/time_vector.rs Outdated Show resolved Hide resolved
@thatzopoulos thatzopoulos force-pushed the th/time-vector-template-json branch 5 times, most recently from 6f524c6 to 10249e4 Compare November 30, 2022 22:39
);

let test_user_supplied_template = client.select(
"SELECT toolkit_experimental.format_timevector(timevector(time, value),'{\"times\": {{ TIMES | json_encode(pretty=true) | safe }}, \"vals\": {{ VALUES | json_encode(pretty=true) | safe }}}') FROM data",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the json_encode(pretty=true) | safe required to make this look like valid JSON, can we do that by default (probably not pretty=true by default though)?

Copy link
Contributor Author

@thatzopoulos thatzopoulos Dec 5, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

json_encode() is required but pretty=true isn't so I can turn that off if we want to.
If you were to write this instead of using json_encode:
{\"times\": {{ TIMES }}, \"vals\": {{ VALUES }}} it would render as:

`"{\"times\": [2020-01-01 00:00:00+00, 2020-01-02 00:00:00+00, 2020-01-03 00:00:00+00, 2020-01-04 00:00:00+00, 2020-01-05 00:00:00+00], \"vals\": [30, 45, null, 55.5, 10]}"

json_encode without pretty printing:

{\"times\": {{ TIMES | json_encode() | safe  }}, \"vals\": {{ VALUES | json_encode() | safe }}}'

looks like this:

"{\"times\": [\"2020-01-01 00:00:00+00\",\"2020-01-02 00:00:00+00\",\"2020-01-03 00:00:00+00\",\"2020-01-04 00:00:00+00\",\"2020-01-05 00:00:00+00\"], \"vals\": [\"30\",\"45\",\"null\",\"55.5\",\"10\"]}"`

There doesn't seem to be a good builtin way to run json_encode() on all user template code by default though. This might be something we can investigate in a future improvement?

extension/src/time_vector.rs Show resolved Hide resolved
extension/src/time_vector.rs Outdated Show resolved Hide resolved
extension/src/time_vector.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@WireBaron WireBaron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is in a good state for experimental release, but I would like us to have a good solution for users who want a TIMEVALs vector without having to learn Tera formatting.

extension/src/time_vector.rs Show resolved Hide resolved
context.insert("TIMES", &times);
context.insert("VALUES", &values);

context.insert("TIMEVALS", &timevals); // paired timevals: ex. (time,value).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure either of the representations here (line 171 and 172) are that useful, we'd really like to see this output a valid JSON vector of time, value structs.

I'm guessing we can't just specify a format string like "{\"times\": {{ TIMES | json_encode(pretty=true) | safe }}, \"vals\": {{ VALUES | json_encode(pretty=true) | safe }}}" here? Could we just parse the string ourselves and replace occurrences of TIMEVAL with that string?

Copy link
Contributor Author

@thatzopoulos thatzopoulos Dec 5, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense, I am currently trying to get that first idea to work and if it does not I will try out the second approach.
I got the first approach working once I turned off Tera's autoescape. This should work now:

SELECT toolkit_experimental.to_text(timevector(time, value),'{{TIMEVALS}}') FROM data
[{\"time\": \"2020-01-01 00:00:00+00\", \"val\": 30}, {\"time\": \"2020-01-02 00:00:00+00\", \"val\": 45}, {\"time\": \"2020-01-03 00:00:00+00\", \"val\": null}, {\"time\": \"2020-01-04 00:00:00+00\", \"val\": 55.5}, {\"time\": \"2020-01-05 00:00:00+00\", \"val\": 10} ]

@thatzopoulos thatzopoulos force-pushed the th/time-vector-template-json branch 2 times, most recently from bf07375 to 55e8135 Compare December 5, 2022 22:42
@thatzopoulos
Copy link
Contributor Author

bors r+

@bors
Copy link
Contributor

bors bot commented Dec 6, 2022

@bors bors bot merged commit c9d6331 into main Dec 6, 2022
@bors bors bot deleted the th/time-vector-template-json branch December 6, 2022 18:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants