Parquet multi kernel optimization example.

[nvdbaranec]

This PR is intended to be a concrete implementation of the multi-kernel parquet decode concept. Not necessarily to be checked in in this current form. In short: having large monolithic kernels that can decode all permutations of page data (fixed-width, variable-width, nested, dictionaries, etc) results in poor performance. Increased worst-case shared memory usage, higher per-thread register counts, etc. I spent some time working on an optimization that made the decoder work in a more fundamentally block-wide way, but this ended up not being too much of a win for the above state reasons. My initial pass was quite fast and then as I slowly added new features it ground back down.

So the fundamental idea here is to break up the various types of data in the pages of a parquet file and use stripped down kernels specific to each type. Each kernel is launched on a separate, non-default stream to ensure overlapping. There are several benefits to doing it this way:

• Each individual kernel can be launched with it's own set of grid parameters. For example, fixed-width decoders might perform better with larger blocks than variable-width decoders (or vice-versa).
• Shared memory usage can be dramatically dropped and limited to exactly what is needed for a given kernel.
• The slimmer kernels can flow around the bigger ones as hardware becomes available. With the monolithic kernel, the single grid size guarantees you will always be running as the biggest, heaviest kernel we have.

This PR does several things:

• It breaks up page_data.cu into several component pieces. decode.cuh represents common code shared between the various kernels. decode_general.cu contains the existing, monolithic kernel we already use. decode_fixed.cu contains an optimized version of the decode kernel that targets fixed-width, non-nested, non-dictionaried data.
• The new kernel uses the rle_stream to implement a block-wide decoding scheme, instead of the overlapped-multiple-warp technique of the existing one.
• At header decode time, each page is assigned a mask value (via get_kernel_mask) that represents the specific kernel it should be decoded with. At data decode time, we or all of the required kernel bits for all the pages together, and launch which kernels we need (see DecodePageData in page_data.cu)
• Many functions and structures have been templatized to account for the variable grid sizes and shared memory requirements.

General performance is very good for this data subset: roughly 50% speedup across the board in the decode step. This does not lead to that level of speedup for the cudf benchmarks for 2 main reasons:

• They tend to spend about half their time in nvcomp decompression.
• Many of the benchmarks used a wide variety of data types, so this new kernel is only invoked on a subset of the data.

Nonetheless in situations where we have little or no compression, the wins are clear:

Before

| data_type |      io       | bytes_per_second | encoded_file_size |
|     FLOAT | DEVICE_BUFFER |  35470873266     |       510.303 MiB |

After

| data_type |      io       | bytes_per_second | encoded_file_size |
|     FLOAT | DEVICE_BUFFER | 54260511096      |       510.303 MiB |

For cases where we have to go through nvcomp and we're dealing with a mix of new and old kernels, the wins are more mild, even though the decode step itself is still in the 50% speedup ballpark. Below is an example with integers.

Before. Total load time was 37. Only 18ms of that was the parquet decoder.

After. Total load time was 29ms. But only 10.5 of that was the parquet decoder, with a mix of the new and old kernels.

So a total time decode time reduction from 18ms to 10.5 ms.

You may notice that in the "after" case, there does not appear to be any kernel overlap. There's two things going on here. First, the fixed-width kernel runs at 100% occupancy, so there's no unused hardware. Second, you can see that when it is near completion, the general purpose kernel starts to flow in (and also runs at considerably lower occupancy)

Related: I have a branch up (not an actual PR) with the full heavyweight implementation of all features in the block-wide style. Hypothetically, all we need to do is cherry pick out the individual chunks of logic from there to build more kernels. This branch also includes a change to rle_stream that distributes the level-decoding work for arbitrarily sized runs evenly across all warps instead of the greedy one-warp-per-run approach.

https://github.com/nvdbaranec/cudf/tree/parquet_decode_optimization
https://github.com/nvdbaranec/cudf/blob/parquet_decode_optimization/cpp/src/io/parquet/page_data.cu

[nvdbaranec]

Open questions:

• We would need to figure out what the right subset of kernels actually is. Based on our feature set, it's easy to imagine 16+ kernels with slightly different features. But that might be overkill. I tend to think the primary way of breaking them down should be based on shared memory usage differences.

• There's more optimization that could be pushed through here. For example, the individual "copy this piece of data" functions are shared between the new and old kernels, but they all do a dictionary if-check. This could be if constexpr'd away if we wanted to go crazier.

• This file layout is just a suggestion. page_data.cu was getting pretty crazy so I wanted to try out some ideas on splitting things up.

[nvdbaranec]

Also, there will be a decent amount of work to get this merged with the big string decoding change from @etseidl

[etseidl]

I've been thinking about this one. Would it be better to break up pages into separate batches grouped by kernel_mask? That way, each batch could make it's own way through pre-process and process, with each batch getting its own stream. Maybe fixed-width decode logic could overlap with some of the string preprocessing, for instance.

[GregoryKimball]

FYI adding specialized decode kernels to parquet reader came up again today in a discussion about the query performance roadmap. I believe the next step was to group the different data types, nullability and encodings by register usage. Then from there we could start adding specialized kernels one at a time.

Please let me know if you agree.

[nvdbaranec]

Obsolete with #15159