Off-by-one error in year-month timeUnit

[jakevdp]

The following specification has dates starting and ending in the month of June, but the chart renders with data starting and ending in the month of May. Reported in vega/altair#1027

{
  "data": {
    "values": [
      {"date": "2017-06-01", "pct_change_rounded": 0},
      {"date": "2017-07-01", "pct_change_rounded": 0.1},
      {"date": "2017-08-01", "pct_change_rounded": 0.4},
      {"date": "2017-09-01", "pct_change_rounded": 0.5},
      {"date": "2017-10-01", "pct_change_rounded": 0.1},
      {"date": "2017-11-01", "pct_change_rounded": 0.3},
      {"date": "2017-12-01", "pct_change_rounded": 0.2},
      {"date": "2018-01-01", "pct_change_rounded": 0.5},
      {"date": "2018-02-01", "pct_change_rounded": 0.2},
      {"date": "2018-03-01", "pct_change_rounded": -0.1},
      {"date": "2018-04-01", "pct_change_rounded": 0.2},
      {"date": "2018-05-01", "pct_change_rounded": 0.2},
      {"date": "2018-06-01", "pct_change_rounded": 0.1}
    ]
  },
  "mark": "bar",
  "encoding": {
    "x": {
      "type": "ordinal",
      "axis": {"format": "%b %y"},
      "field": "date",
      "timeUnit": "yearmonth"
    },
    "y": {"type": "quantitative", "field": "pct_change_rounded"}
  }
}

[kanitw]

This is because Javascript parses time in an ISO into UTC time.
Since Vega-Lite doesn't look at the data, we only always output local time by default even if the input is in UTC time.

If you use utcyearmonth, then this would work correctly.

{
  "data": {
    "values": [
      {"date": "2017-06-01", "pct_change_rounded": 0},
      {"date": "2017-07-01", "pct_change_rounded": 0.1},
      {"date": "2017-08-01", "pct_change_rounded": 0.4},
      {"date": "2017-09-01", "pct_change_rounded": 0.5},
      {"date": "2017-10-01", "pct_change_rounded": 0.1},
      {"date": "2017-11-01", "pct_change_rounded": 0.3},
      {"date": "2017-12-01", "pct_change_rounded": 0.2},
      {"date": "2018-01-01", "pct_change_rounded": 0.5},
      {"date": "2018-02-01", "pct_change_rounded": 0.2},
      {"date": "2018-03-01", "pct_change_rounded": -0.1},
      {"date": "2018-04-01", "pct_change_rounded": 0.2},
      {"date": "2018-05-01", "pct_change_rounded": 0.2},
      {"date": "2018-06-01", "pct_change_rounded": 0.1}
    ]
  },
  "mark": "bar",
  "encoding": {
    "x": {
      "type": "ordinal",
      "axis": {"format": "%b %y"},
      "field": "date",
      "timeUnit": "utcyearmonth"
    },
    "y": {"type": "quantitative", "field": "pct_change_rounded"}
  }
}

[kanitw]

This is a bit annoying, but there is no good way to solve this in Vega-Lite unless we revamp how Vega works with time.

[jakevdp]

Thanks

[ijlyttle]

Hi all,

I'm not sure if this is the right place to start such a conversation (maybe you can direct me), but how feasible might it be to revamp how Vega deals with time?

My motivation is that I am trying to get my company to adopt Vega/Vega-Lite, but we are an international company, with international customers. For example, it would not be unusual for someone in "Europe/Paris" to want to see data in the context of "America/New_York", regardless of the timezone of their browser.

Accordingly, I see these sorts of time-zone issues as being the principal barrier to eventual widespread adoption within my company.

I know this is not a small problem to tackle, so I do not bring this up lightly. Of course, I would be happy to pitch in. Although I (currently) lack in JavaScript skills, addressing this issue would motivate me to (try to) get up to speed.

I am happy to continue this conversation wherever. This is the sort if thing I can (pardon the pun) make the time to participate in.

Thanks!

[domoritz]

We're having a discussion about this in the vega slack in #altair.

The short story is that JavaScript date parsing is odd and it's hard to get around that.

[domoritz]

Also vega/altair#1027

[jakevdp]

@ijlyttle are you creating charts from Vega-Lite directly, or using Altair?

[jakevdp]

I think the solution is to use fully-specified ISO-8601 dates everywhere.

[ijlyttle]

I have been creating using Altair, but I have been careful (or at least I think I have been careful) to use ISO-8601 whenever I can.

I agree that ISO-8601 is the only way to maintain sanity 😬

Let me see what I can reproduce.

[jakevdp]

If you're using Altair with Pandas dataframes & datetimes, then you haven't been using ISO and I think the change I have in mind at vega/altair#1027 will fix things for you.

[ijlyttle]

@jakevdp Just to say thanks for the conversations across the various issues. I am glad to see Altair moving towards ISO formatting.

I want to think about different ways of handling time when building a spec - your local-time way being one of them, and to open an issue to see it there's a way to support many ways of doing things. I think I want to take some time to go through all of the slack and github conversations first, to make sure I get all the different perspectives.

[jakevdp]

Thanks @ijlyttle – and thanks for your comments and feedback.

One thing that would be worth thinking about is whether we can support datetime64[ns,utc] as well as datetime64[ns] in a streamlined way. That way people could choose UTC or not within Python, and everything else should just flow from there.

[iliatimofeev]

whether we can support datetime64[ns,utc]

I guess Altair is already support timestamp with timezone including 'utc'. vega/altair#926

[jakevdp]

It supports them in the sense that it doesn't raise an error if you use UTC dates. I'm talking about supporting them in the sense of propagating the UTC property through to the vega-lite spec.

[mattijn]

Little comment regarding timezones: probably best bet to have timezones based on a package that implements the Olson/IANA timezones database (like luxon or moment.js), but realise that some timezones still using POSIX standard and not ISO 8601. Which I faced last week in Python: https://stackoverflow.com/q/51542167/2459096

…

On Sat, 28 Jul 2018 at 03:07, Jake Vanderplas ***@***.***> wrote: It supports them in the sense that it doesn't raise an error if you use UTC dates. I'm talking about supporting them in the sense of propagating the UTC property through to the vega-lite spec. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#4044 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AE8i2VTEnp85UCvG0s0xZPvFQNAaHvg_ks5uK7lggaJpZM4VTAPF> .

[jakevdp]

Fix for this in Altair: vega/altair#1053