Add packed serialization option for dataframes

[charlesbluca]

Adds the option to enable pack/unpack for dataframe serialization using the environment variable CUDF_PACKED_SERIALIZATION. This is intended to be a temporary solution to aid in benchmarking, which will eventually be replaced by a config module discussed in #5311.

[brandon-b-miller]

@charlesbluca this looks fine overall but I am wondering how we plan on testing this? Seems we have some tests for serialization that might suffice from the perspective of coverage, but you'd have to inject the environment variable in both the true and false state during the test session somehow.

[charlesbluca]

That's a good point I didn't consider - I imagine we could either

• Make a function/context manager to specific enable/disable pack/unpack independent of the env variable
• Call os.environ.get() on each call of serialize/deserialize, from which we should be able to use os.environ.set() to enable pack/unpack from the test script

[brandon-b-miller]

Something like this may help https://docs.pytest.org/en/latest/how-to/monkeypatch.html

[codecov]

Codecov Report

Merging #8661 (db02ffb) into branch-21.08 (3ee264c) will increase coverage by 0.02%.
The diff coverage is 52.94%.

❗ Current head db02ffb differs from pull request most recent head b85966d. Consider uploading reports for the commit b85966d to get more accurate results

@@               Coverage Diff                @@
##           branch-21.08    #8661      +/-   ##
================================================
+ Coverage         10.60%   10.62%   +0.02%     
================================================
  Files               109      109              
  Lines             18280    18292      +12     
================================================
+ Hits               1938     1943       +5     
- Misses            16342    16349       +7     

Impacted Files	Coverage Δ
python/cudf/cudf/core/dataframe.py	0.00% <0.00%> (ø)
python/dask_cudf/dask_cudf/groupby.py	91.47% <90.00%> (+0.19%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3ee264c...b85966d. Read the comment docs.

[charlesbluca]

Converting this to draft as there are currently a lot of issues with pack/unpack serialization that need to be worked out before this option is really useful for its intended purpose (benchmarking).

[galipremsagar]

Moving this to 21.12, please target this PR to 21.12.

[charlesbluca]

Closing this as we haven't really found a place for pack/unpack serialization yet - will reopen if this changes in the future

[jakirkham]

Now that we have a way to config cuDF ( #11193 ), is this worth revisiting?