Add polygon rendering support based on spatialpandas extension arrays

[jonmmease]

Overview

This PR supersedes #819, adding Polygon rendering support. Whereas #819 included extension arrays that were subclasses of the existing RaggedArray extension array, this PR relies on a family of geometry extensions arrays in the not-yet-published spatialpandas package. To try out this branch, you'll need holoviz/spatialpandas#2 from the spatialpandas repository as well.

ExtensionArray Design

After a lot of prototyping, I ended up having the spatialpandas extension arrays depend on pyarrow, but not fletcher. Fletcher works great, but I found there was a little bit of an impedance mismatch with this usecase. The main reason I think is that for the geometry extension arrays, pyarrow is basically an implementation detail and I want the individual elements to be wrapped in custom classes when they are accessed as scalars. I got pretty far down this road using FletcherArray subclasses, but I had trouble chasing down the edge cases needed to get the pandas extension array test suite passing.

This may be something worth unifying with fletcher in the future (and doing so wouldn't be a breaking change), but I think this is the best first stage given current timeline/constraints.

Examples

I've created a new example notebook for polygon/line/points rendering at https://anaconda.org/jonmmease/datashader_spatialpandas/notebook.

[jbednar]

Looks great! Feel free to merge it once spatialpandas is available for conda installation (e.g. on the pyviz channel).

[jbednar]

For the attached notebook, can you add points examples somewhere, for the same data already shown as polygons and lines? From the source code it looks like points are supported, but I don't see an example of that anymore.

[jonmmease]

This PR is now based on spatialpandas 0.1.0rc1, which has been published to PyPI. Tests won't pass until spatialpandas is available on the pyviz channel.

[jonmmease]

spatialpandas 0.1.0 has been released on PyPI and the pyviz anaconda channel, and CI tests are now passing. PR is ready for final testing/consideration @jbednar @philippjfr

[jbednar]

Looks great! Happy to merge when @philippjfr is done reviewing it.

[philippjfr]

One comment otherwise this looks good and works great.

[jonmmease]

Thanks @philippjfr, I'm looking forward to seeing what you can do with this in HoloViews 🙂

Docstring and merge conflict fixed, so I think this is ready to merge after we push out the release with GPU support.

[philippjfr]

I'll push my HoloViews implementation later today. There's some trickiness around nested holes and detecting CW vs. CCW that I haven't solved yet and which I may ask you about.

[philippjfr]

So I haven't yet worked out whether this is a quirk of the data or an issue rasterizing, but in certain scenarios you can get weird artifacts when rendering polygons:

import geopandas as gpd
import datashader as ds
from spatialpandas import GeoDataFrame

world_gp = gpd.read_file(
    gpd.datasets.get_path('naturalearth_lowres')
)
world_gp = world_gp.sort_values('name').reset_index(drop=True)
world_gp = world_gp[['pop_est', 'continent', 'name', 'geometry']]
spdf = GeoDataFrame(world_gp)

cvs = ds.Canvas(plot_height=245,plot_width=215, x_range=(-600.2587977094785, 720.6321972454359),y_range=(-330.57983377255783, 306.5487454907058))
ds.transfer_functions.shade(cvs.polygons(spdf, 'geometry'))

[jonmmease]

thanks for creating the example, looks like a rendering bug to me, i’ll take a look.

[jonmmease]

@philippjfr, ok I tracked down the problem and fixed it in 6a3c737.

This PR is good-to-go from my end now.

[jbednar]

Beautiful, thanks!

[saulshanabrook]

Super excited about this work! Am I right in thinking this will end up showing up at some point in the hvplot method to pandas? Does that already work?

Or if not, is there a way to use this to create interactive geo viz?

[jbednar]

Yes, that's the idea. First we need the HoloViews support to be merged (holoviz/holoviews#4120), then hvplot will need a trivial extension to support spatialpandas data types (holoviz/hvplot#389). Should come together soon!

[saulshanabrook]

@jbednar Yay! Thanks for the pointers to those PRs, ill follow them. Thanks again everyone who is helping make this happen. It's awesome to have tools like this you can use for climate change research and also for finding a new house :)