Accelerate data transfer for map Pandas UDF plan

[firestarman]

This PR is to accelerate the data transfer between JVM and Python for the plan GpuMapInPandas, by implementing the columnar way.

Also add the related integration tests.

For issue #305

[firestarman]

Will run some benchmark locally before marking ready for review.

[firestarman]

build

[firestarman]

Will run some benchmark locally before marking ready for review.

Got some positive numbers and tests passed on databricks. Mark as ready for review.

[jlowe]

Got some positive numbers

Can you elaborate here what those numbers were? What was the use-case and how much improvement did you see?

[jlowe]

I'll defer to @revans2 for reviewing the details of the iterator changes since he's much more familiar with that code, but it does seem like we could do a better job of testing the various type combinations.

[firestarman]

I'll defer to @revans2 for reviewing the details of the iterator changes since he's much more familiar with that code, but it does seem like we could do a better job of testing the various type combinations.

Thanks for review. I will try to cover more supported types.

[firestarman]

Got some positive numbers

Can you elaborate here what those numbers were? What was the use-case and how much improvement did you see?

Not easy to show the numbers here, so here is the link.

I just ran some benchmark for data transfer locally to verify the columnar way will get better perf, and this PR is for the issue #305 .

[firestarman]

Forced push is only for the commit signature issue. Since the latest merge commit has no signature by default. I added it by a forced push.

[firestarman]

build

[firestarman]

@revans2 Hi, Could you take a look at this PR ? Thanks in advance.

[firestarman]

I am going to merge this, if any concern, I will make new PRs.