Update explain format to show what will and will not run on the GPU

[revans2]

This fixes #906

Plans now look something like

    *Exec <ProjectExec> will run on GPU
      !Exec <FilterExec> cannot run on GPU because not all expressions can be replaced
        @Expression <And> (isnotnull(sum((ps_supplycost#95 * cast(ps_availqty#94L as double)))#1128) AND (sum((ps_supplycost#95 * cast(ps_availqty#94L as double)))#1128 > Subquery scalar-subquery#1122, [id=#14126])) could run on GPU
          @Expression <IsNotNull> isnotnull(sum((ps_supplycost#95 * cast(ps_availqty#94L as double)))#1128) could run on GPU
            @Expression <AttributeReference> sum((ps_supplycost#95 * cast(ps_availqty#94L as double)))#1128 could run on GPU
          @Expression <GreaterThan> (sum((ps_supplycost#95 * cast(ps_availqty#94L as double)))#1128 > Subquery scalar-subquery#1122, [id=#14126]) could run on GPU
            @Expression <AttributeReference> sum((ps_supplycost#95 * cast(ps_availqty#94L as double)))#1128 could run on GPU
            !NOT_FOUND <ScalarSubquery> Subquery scalar-subquery#1122, [id=#14126] cannot run on GPU because no GPU enabled version of expression class org.apache.spark.sql.execution.ScalarSubquery could be found

and

          *Exec <SortMergeJoinExec> will run on GPU
            #Exec <SortExec> could run on GPU but is going to be removed because removing SortExec as part replacing sortMergeJoin with shuffleHashJoin
              #Expression <SortOrder> s_suppkey#108L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed
              *Exec <ShuffleExchangeExec> will run on GPU
                *Partitioning <HashPartitioning> will run on GPU
                *Exec <FilterExec> will run on GPU
                  *Expression <IsNotNull> isnotnull(s_nationkey#111L) will run on GPU
                  *Exec <FileSourceScanExec> will run on GPU

Now

• * indicates things that will run on the GPU
• @ indicates things that could run on the GPU but will not because the Exec they are a part of will not run on the GPU
• # indicates things that have been removed from the plan for some reason
• ! indicates things that cannot run on the GPU

There is no real documentation for this format so I didn't update it. If this is something that we want to add I am happy to do it.

[revans2]

build

[jlowe]

Looks good overall, but should the user docs be updated to explain the meaning of the new symbols that are being added in the explain logging?

[revans2]

Looks good overall, but should the user docs be updated to explain the meaning of the new symbols that are being added in the explain logging?

I am happy to do it, but I am not sure where to do it. Like I said there is no documentation right now explaining the format.

I could update it in configs.md, but I am not sure that is the best place. The only other place it is mentioned is in tuning-guide.md

[tgravescs]

I would think putting something in FAQ section and then also in the Monitoring section of the getting started on prep guide would be useful. but having an explanation be in its own section somewhere and link to it from those places makes sense to me. Or maybe we need to make the monitoring section its own.

[jlowe]

putting something in FAQ section

+1 for this. I think minimally the FAQ should have an entry for, "How do I tell if my query is running on the GPU?" which arguably is our most asked question. The FAQ also does mention explain in the first entry.

[revans2]

I added docs to the FAQ.

[revans2]

build