SPARK-1093: Annotate developer and experimental API's #274

pwendell · 2014-03-31T00:38:44Z

This patch marks some existing classes as private[spark] and adds two types of API annotations:

EXPERIMENTAL API = experimental user-facing module
DEVELOPER API - UNSTABLE = developer-facing API that might change

There is some discussion of the different mechanisms for doing this here:
https://issues.apache.org/jira/browse/SPARK-1081

I was pretty aggressive with marking things private. Keep in mind that if we want to open something up in the future we can, but we can never reduce visibility.

A few notes here:

In the past we've been inconsistent with the visiblity of the X-RDD classes. This patch marks them private whenever there is an existing function in RDD that can directly creat them (e.g. CoalescedRDD and rdd.coalesce()). One trade-off here is users can't subclass them.
Noted that compression and serialization formats don't have to be wire compatible across versions.
Compression codecs and serialization formats are semi-private as users typically don't instantiate them directly.
Metrics sources are made private - user only interacts with them through Spark's reflection

AmplabJenkins · 2014-03-31T00:42:22Z

Merged build triggered. Build is starting -or- tests failed to complete.

AmplabJenkins · 2014-03-31T00:42:28Z

Merged build started. Build is starting -or- tests failed to complete.

AmplabJenkins · 2014-03-31T01:39:03Z

Merged build finished. All automated tests passed.

AmplabJenkins · 2014-03-31T01:39:03Z

All automated tests passed.
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13589/

mateiz · 2014-04-03T00:41:17Z

Hey Patrick, a few comments:

Should Logging become private[spark]? I think we looked at it before and there was some weird reason why we didn't do it, not sure what.
The badges unfortunately display weirdly in summary pages. For example look at this one for the root package. The first badge there is actually for Aggregator, but it displays on the bottom of its line, which makes it look like it might belong to the next element.
It might be good to give these different colors, e.g. something blue for Experimental API.

mateiz · 2014-04-03T00:54:03Z

Some other classes that may need to be annotated:

InterruptibleIterator -- probably private[spark]
RDD.*Approx (e.g. countApprox), and similar methods in JavaRDD, DoubleRDD, PairRDD -- experimental
RDD.mapPartitionsWithContext -- developer
JobLogger -- developer (I also saw it's deprecated)
scheduler.SchedulingMode -- private[spark]
scheduler.SplitInfo -- either private[spark] or developer (if it appears in events)
storage.StoragePerfTester -- should probably just be moved to the Spark Tools project
util.Vector -- deprecate this, MLlib will have its own, better ones
util.BoundedPriorityQueue -- any reason this is not just private[spark]?
SimpleFutureAction -- developer
SparkContext.addSparkListener -- developer
SparkContext.runApproximateJob -- developer
SparkContext.runJob -- developer
SparkContext.submitJob -- experimental
SparkContext.warnSparkMem -- probably need to be private
broadcast.BroadcastFactory -- developer
RDD.compute, dependencies, iterator and such should probably also be developer

mateiz · 2014-04-03T00:56:56Z

BTW to fix the floating badge problem, you might do the following: change code like this:

  <span class="badge badge-red" style="float: right;">DEVELOPER API - UNSTABLE</span>

   Represents a one-to-one dependency between partitions of the parent and child RDDs.

To this:

  <span class="badge badge-red" style="float: right;">DEVELOPER API - UNSTABLE
  </span>Represents a one-to-one dependency between partitions of the parent and child RDDs.

I believe Scaladoc includes only the first sentence it finds, so this might make it include both the text and the floating span. It might also not work though. But the reason those are displaying weirdly is partly that the first line there is ignored.

rxin · 2014-04-03T08:56:20Z

core/src/main/scala/org/apache/spark/rdd/CoGroupedRDD.scala

@@ -57,6 +57,7 @@ private[spark] class CoGroupPartition(idx: Int, val deps: Array[CoGroupSplitDep]
 * @param rdds parent RDDs.
 * @param part partitioner used to partition the shuffle output.
 */
+private[spark]


You might want to relax this since I don't think the user cannot construct Product2 version of the CoGroupedRDD in PairRDDFunctions

AmplabJenkins · 2014-04-05T23:52:23Z

Build triggered.

AmplabJenkins · 2014-04-05T23:52:32Z

Build started.

AmplabJenkins · 2014-04-05T23:54:35Z

Build finished.

AmplabJenkins · 2014-04-05T23:54:35Z

Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13806/

AmplabJenkins · 2014-04-05T23:57:26Z

Build triggered.

AmplabJenkins · 2014-04-05T23:57:32Z

Build started.

AmplabJenkins · 2014-04-06T00:00:20Z

Build finished.

aarondav · 2014-04-09T05:30:40Z

I vote for spark.annotation, package names are generally singular, e.g., "javax.annotation" which I think trumps Hadoop any day.

mengxr · 2014-04-09T05:30:51Z

That is the artifact name. Java uses annotation: http://docs.oracle.com/javase/7/docs/api/java/lang/annotation/Documented.html

pwendell · 2014-04-09T05:32:55Z

Okay - I'll change this to annotation

AmplabJenkins · 2014-04-09T05:37:23Z

Build triggered.

AmplabJenkins · 2014-04-09T05:37:30Z

Build started.

AmplabJenkins · 2014-04-09T06:34:06Z

Build finished. All automated tests passed.

AmplabJenkins · 2014-04-09T06:34:06Z

All automated tests passed.
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13931/

Conflicts: core/src/main/scala/org/apache/spark/scheduler/JobResult.scala core/src/main/scala/org/apache/spark/storage/StorageUtils.scala core/src/main/scala/org/apache/spark/util/TimeStampedHashMap.scala sql/core/src/main/scala/org/apache/spark/sql/SchemaRDD.scala

AmplabJenkins · 2014-04-09T07:22:23Z

Merged build triggered.

AmplabJenkins · 2014-04-09T07:22:31Z

Merged build started.

AmplabJenkins · 2014-04-09T08:01:10Z

Merged build finished. All automated tests passed.

AmplabJenkins · 2014-04-09T08:01:10Z

All automated tests passed.
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13939/

pwendell · 2014-04-09T08:16:26Z

Just merged this.

@DeveloperAPI

... so that we don't follow an unspoken set of forbidden rules for adding **@AlphaComponent**, **@DeveloperAPI**, and **@experimental** annotations in the code. In addition, this PR (1) removes unnecessary `:: * ::` tags, (2) adds missing `:: * ::` tags, and (3) removes annotations for internal APIs. Author: Andrew Or <andrewor14@gmail.com> Closes #470 from andrewor14/annotations-fix and squashes the following commits: 92a7f42 [Andrew Or] Document + fix annotation usages

@DeveloperAPI

... so that we don't follow an unspoken set of forbidden rules for adding **@AlphaComponent**, **@DeveloperAPI**, and **@experimental** annotations in the code. In addition, this PR (1) removes unnecessary `:: * ::` tags, (2) adds missing `:: * ::` tags, and (3) removes annotations for internal APIs. Author: Andrew Or <andrewor14@gmail.com> Closes #470 from andrewor14/annotations-fix and squashes the following commits: 92a7f42 [Andrew Or] Document + fix annotation usages (cherry picked from commit b3e5366) Signed-off-by: Patrick Wendell <pwendell@gmail.com>

This patch marks some existing classes as private[spark] and adds two types of API annotations: - `EXPERIMENTAL API` = experimental user-facing module - `DEVELOPER API - UNSTABLE` = developer-facing API that might change There is some discussion of the different mechanisms for doing this here: https://issues.apache.org/jira/browse/SPARK-1081 I was pretty aggressive with marking things private. Keep in mind that if we want to open something up in the future we can, but we can never reduce visibility. A few notes here: - In the past we've been inconsistent with the visiblity of the X-RDD classes. This patch marks them private whenever there is an existing function in RDD that can directly creat them (e.g. CoalescedRDD and rdd.coalesce()). One trade-off here is users can't subclass them. - Noted that compression and serialization formats don't have to be wire compatible across versions. - Compression codecs and serialization formats are semi-private as users typically don't instantiate them directly. - Metrics sources are made private - user only interacts with them through Spark's reflection Author: Patrick Wendell <pwendell@gmail.com> Author: Andrew Or <andrewor14@gmail.com> Closes apache#274 from pwendell/private-apis and squashes the following commits: 44179e4 [Patrick Wendell] Merge remote-tracking branch 'apache-github/master' into private-apis 042c803 [Patrick Wendell] spark.annotations -> spark.annotation bfe7b52 [Patrick Wendell] Adding experimental for approximate counts 8d0c873 [Patrick Wendell] Warning in SparkEnv 99b223a [Patrick Wendell] Cleaning up annotations e849f64 [Patrick Wendell] Merge pull request apache#2 from andrewor14/annotations 982a473 [Andrew Or] Generalize jQuery matching for non Spark-core API docs a01c076 [Patrick Wendell] Merge pull request apache#1 from andrewor14/annotations c1bcb41 [Andrew Or] DeveloperAPI -> DeveloperApi 0d48908 [Andrew Or] Comments and new lines (minor) f3954e0 [Andrew Or] Add identifier tags in comments to work around scaladocs bug 99192ef [Andrew Or] Dynamically add badges based on annotations 824011b [Andrew Or] Add support for injecting arbitrary JavaScript to API docs 037755c [Patrick Wendell] Some changes after working with andrew or f7d124f [Patrick Wendell] Small fixes c318b24 [Patrick Wendell] Use CSS styles e4c76b9 [Patrick Wendell] Logging f390b13 [Patrick Wendell] Better visibility for workaround constructors d6b0afd [Patrick Wendell] Small chang to existing constructor 403ba52 [Patrick Wendell] Style fix 870a7ba [Patrick Wendell] Work around for SI-8479 7fb13b2 [Patrick Wendell] Changes to UnionRDD and EmptyRDD 4a9e90c [Patrick Wendell] EXPERIMENTAL API --> EXPERIMENTAL c581dce [Patrick Wendell] Changes after building against Shark. 8452309 [Patrick Wendell] Style fixes 1ed27d2 [Patrick Wendell] Formatting and coloring of badges cd7a465 [Patrick Wendell] Code review feedback 2f706f1 [Patrick Wendell] Don't use floats 542a736 [Patrick Wendell] Small fixes cf23ec6 [Patrick Wendell] Marking GraphX as alpha d86818e [Patrick Wendell] Another naming change 5a76ed6 [Patrick Wendell] More visiblity clean-up 42c1f09 [Patrick Wendell] Using better labels 9d48cbf [Patrick Wendell] Initial pass

@DeveloperAPI

... so that we don't follow an unspoken set of forbidden rules for adding **@AlphaComponent**, **@DeveloperAPI**, and **@experimental** annotations in the code. In addition, this PR (1) removes unnecessary `:: * ::` tags, (2) adds missing `:: * ::` tags, and (3) removes annotations for internal APIs. Author: Andrew Or <andrewor14@gmail.com> Closes apache#470 from andrewor14/annotations-fix and squashes the following commits: 92a7f42 [Andrew Or] Document + fix annotation usages

[NOSQUASH] Resync Apache

Add huaweicloud logic for export-cloud-openrc

…runing ### What changes were proposed in this pull request? Remove `OptimizeSubqueries` from batch of `PartitionPruning` to make DPP support more cases. For example: ```sql SELECT date_id, product_id FROM fact_sk f JOIN (select store_id + 3 as new_store_id from dim_store where country = 'US') s ON f.store_id = s.new_store_id ``` Before this PR: ``` == Physical Plan == *(2) Project [date_id#3998, product_id#3999] +- *(2) BroadcastHashJoin [store_id#4001], [new_store_id#3997], Inner, BuildRight, false :- *(2) ColumnarToRow : +- FileScan parquet default.fact_sk[date_id#3998,product_id#3999,store_id#4001] Batched: true, DataFilters: [], Format: Parquet, PartitionFilters: [isnotnull(store_id#4001), dynamicpruningexpression(true)], PushedFilters: [], ReadSchema: struct<date_id:int,product_id:int> +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [id=#274] +- *(1) Project [(store_id#4002 + 3) AS new_store_id#3997] +- *(1) Filter ((isnotnull(country#4004) AND (country#4004 = US)) AND isnotnull((store_id#4002 + 3))) +- *(1) ColumnarToRow +- FileScan parquet default.dim_store[store_id#4002,country#4004] Batched: true, DataFilters: [isnotnull(country#4004), (country#4004 = US), isnotnull((store_id#4002 + 3))], Format: Parquet, PartitionFilters: [], PushedFilters: [IsNotNull(country), EqualTo(country,US)], ReadSchema: struct<store_id:int,country:string> ``` After this PR: ``` == Physical Plan == *(2) Project [date_id#3998, product_id#3999] +- *(2) BroadcastHashJoin [store_id#4001], [new_store_id#3997], Inner, BuildRight, false :- *(2) ColumnarToRow : +- FileScan parquet default.fact_sk[date_id#3998,product_id#3999,store_id#4001] Batched: true, DataFilters: [], Format: Parquet, PartitionFilters: [isnotnull(store_id#4001), dynamicpruningexpression(store_id#4001 IN dynamicpruning#4007)], PushedFilters: [], ReadSchema: struct<date_id:int,product_id:int> : +- SubqueryBroadcast dynamicpruning#4007, 0, [new_store_id#3997], [id=#263] : +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [id=#262] : +- *(1) Project [(store_id#4002 + 3) AS new_store_id#3997] : +- *(1) Filter ((isnotnull(country#4004) AND (country#4004 = US)) AND isnotnull((store_id#4002 + 3))) : +- *(1) ColumnarToRow : +- FileScan parquet default.dim_store[store_id#4002,country#4004] Batched: true, DataFilters: [isnotnull(country#4004), (country#4004 = US), isnotnull((store_id#4002 + 3))], Format: Parquet, PartitionFilters: [], PushedFilters: [IsNotNull(country), EqualTo(country,US)], ReadSchema: struct<store_id:int,country:string> +- ReusedExchange [new_store_id#3997], BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [id=#262] ``` This is because `OptimizeSubqueries` will infer more filters, so we cannot reuse broadcasts. The following is the plan if disable `spark.sql.optimizer.dynamicPartitionPruning.reuseBroadcastOnly`: ``` == Physical Plan == *(2) Project [date_id#3998, product_id#3999] +- *(2) BroadcastHashJoin [store_id#4001], [new_store_id#3997], Inner, BuildRight, false :- *(2) ColumnarToRow : +- FileScan parquet default.fact_sk[date_id#3998,product_id#3999,store_id#4001] Batched: true, DataFilters: [], Format: Parquet, PartitionFilters: [isnotnull(store_id#4001), dynamicpruningexpression(store_id#4001 IN subquery#4009)], PushedFilters: [], ReadSchema: struct<date_id:int,product_id:int> : +- Subquery subquery#4009, [id=#284] : +- *(2) HashAggregate(keys=[new_store_id#3997#4008], functions=[]) : +- Exchange hashpartitioning(new_store_id#3997#4008, 5), ENSURE_REQUIREMENTS, [id=#280] : +- *(1) HashAggregate(keys=[new_store_id#3997 AS new_store_id#3997#4008], functions=[]) : +- *(1) Project [(store_id#4002 + 3) AS new_store_id#3997] : +- *(1) Filter (((isnotnull(store_id#4002) AND isnotnull(country#4004)) AND (country#4004 = US)) AND isnotnull((store_id#4002 + 3))) : +- *(1) ColumnarToRow : +- FileScan parquet default.dim_store[store_id#4002,country#4004] Batched: true, DataFilters: [isnotnull(store_id#4002), isnotnull(country#4004), (country#4004 = US), isnotnull((store_id#4002..., Format: Parquet, PartitionFilters: [], PushedFilters: [IsNotNull(store_id), IsNotNull(country), EqualTo(country,US)], ReadSchema: struct<store_id:int,country:string> +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [id=#305] +- *(1) Project [(store_id#4002 + 3) AS new_store_id#3997] +- *(1) Filter ((isnotnull(country#4004) AND (country#4004 = US)) AND isnotnull((store_id#4002 + 3))) +- *(1) ColumnarToRow +- FileScan parquet default.dim_store[store_id#4002,country#4004] Batched: true, DataFilters: [isnotnull(country#4004), (country#4004 = US), isnotnull((store_id#4002 + 3))], Format: Parquet, PartitionFilters: [], PushedFilters: [IsNotNull(country), EqualTo(country,US)], ReadSchema: struct<store_id:int,country:string> ``` ### Why are the changes needed? Improve DPP to support more cases. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Unit test and benchmark test: SQL | Before this PR(Seconds) | After this PR(Seconds) -- | -- | -- TPC-DS q58 | 40 | 20 TPC-DS q83 | 18 | 14 Closes #33664 from wangyum/SPARK-36444. Authored-by: Yuming Wang <yumwang@ebay.com> Signed-off-by: Yuming Wang <yumwang@ebay.com>

…runing ### What changes were proposed in this pull request? Remove `OptimizeSubqueries` from batch of `PartitionPruning` to make DPP support more cases. For example: ```sql SELECT date_id, product_id FROM fact_sk f JOIN (select store_id + 3 as new_store_id from dim_store where country = 'US') s ON f.store_id = s.new_store_id ``` Before this PR: ``` == Physical Plan == *(2) Project [date_id#3998, product_id#3999] +- *(2) BroadcastHashJoin [store_id#4001], [new_store_id#3997], Inner, BuildRight, false :- *(2) ColumnarToRow : +- FileScan parquet default.fact_sk[date_id#3998,product_id#3999,store_id#4001] Batched: true, DataFilters: [], Format: Parquet, PartitionFilters: [isnotnull(store_id#4001), dynamicpruningexpression(true)], PushedFilters: [], ReadSchema: struct<date_id:int,product_id:int> +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [id=#274] +- *(1) Project [(store_id#4002 + 3) AS new_store_id#3997] +- *(1) Filter ((isnotnull(country#4004) AND (country#4004 = US)) AND isnotnull((store_id#4002 + 3))) +- *(1) ColumnarToRow +- FileScan parquet default.dim_store[store_id#4002,country#4004] Batched: true, DataFilters: [isnotnull(country#4004), (country#4004 = US), isnotnull((store_id#4002 + 3))], Format: Parquet, PartitionFilters: [], PushedFilters: [IsNotNull(country), EqualTo(country,US)], ReadSchema: struct<store_id:int,country:string> ``` After this PR: ``` == Physical Plan == *(2) Project [date_id#3998, product_id#3999] +- *(2) BroadcastHashJoin [store_id#4001], [new_store_id#3997], Inner, BuildRight, false :- *(2) ColumnarToRow : +- FileScan parquet default.fact_sk[date_id#3998,product_id#3999,store_id#4001] Batched: true, DataFilters: [], Format: Parquet, PartitionFilters: [isnotnull(store_id#4001), dynamicpruningexpression(store_id#4001 IN dynamicpruning#4007)], PushedFilters: [], ReadSchema: struct<date_id:int,product_id:int> : +- SubqueryBroadcast dynamicpruning#4007, 0, [new_store_id#3997], [id=#263] : +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [id=#262] : +- *(1) Project [(store_id#4002 + 3) AS new_store_id#3997] : +- *(1) Filter ((isnotnull(country#4004) AND (country#4004 = US)) AND isnotnull((store_id#4002 + 3))) : +- *(1) ColumnarToRow : +- FileScan parquet default.dim_store[store_id#4002,country#4004] Batched: true, DataFilters: [isnotnull(country#4004), (country#4004 = US), isnotnull((store_id#4002 + 3))], Format: Parquet, PartitionFilters: [], PushedFilters: [IsNotNull(country), EqualTo(country,US)], ReadSchema: struct<store_id:int,country:string> +- ReusedExchange [new_store_id#3997], BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [id=#262] ``` This is because `OptimizeSubqueries` will infer more filters, so we cannot reuse broadcasts. The following is the plan if disable `spark.sql.optimizer.dynamicPartitionPruning.reuseBroadcastOnly`: ``` == Physical Plan == *(2) Project [date_id#3998, product_id#3999] +- *(2) BroadcastHashJoin [store_id#4001], [new_store_id#3997], Inner, BuildRight, false :- *(2) ColumnarToRow : +- FileScan parquet default.fact_sk[date_id#3998,product_id#3999,store_id#4001] Batched: true, DataFilters: [], Format: Parquet, PartitionFilters: [isnotnull(store_id#4001), dynamicpruningexpression(store_id#4001 IN subquery#4009)], PushedFilters: [], ReadSchema: struct<date_id:int,product_id:int> : +- Subquery subquery#4009, [id=#284] : +- *(2) HashAggregate(keys=[new_store_id#3997#4008], functions=[]) : +- Exchange hashpartitioning(new_store_id#3997#4008, 5), ENSURE_REQUIREMENTS, [id=#280] : +- *(1) HashAggregate(keys=[new_store_id#3997 AS new_store_id#3997#4008], functions=[]) : +- *(1) Project [(store_id#4002 + 3) AS new_store_id#3997] : +- *(1) Filter (((isnotnull(store_id#4002) AND isnotnull(country#4004)) AND (country#4004 = US)) AND isnotnull((store_id#4002 + 3))) : +- *(1) ColumnarToRow : +- FileScan parquet default.dim_store[store_id#4002,country#4004] Batched: true, DataFilters: [isnotnull(store_id#4002), isnotnull(country#4004), (country#4004 = US), isnotnull((store_id#4002..., Format: Parquet, PartitionFilters: [], PushedFilters: [IsNotNull(store_id), IsNotNull(country), EqualTo(country,US)], ReadSchema: struct<store_id:int,country:string> +- BroadcastExchange HashedRelationBroadcastMode(List(cast(input[0, int, true] as bigint)),false), [id=#305] +- *(1) Project [(store_id#4002 + 3) AS new_store_id#3997] +- *(1) Filter ((isnotnull(country#4004) AND (country#4004 = US)) AND isnotnull((store_id#4002 + 3))) +- *(1) ColumnarToRow +- FileScan parquet default.dim_store[store_id#4002,country#4004] Batched: true, DataFilters: [isnotnull(country#4004), (country#4004 = US), isnotnull((store_id#4002 + 3))], Format: Parquet, PartitionFilters: [], PushedFilters: [IsNotNull(country), EqualTo(country,US)], ReadSchema: struct<store_id:int,country:string> ``` ### Why are the changes needed? Improve DPP to support more cases. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Unit test and benchmark test: SQL | Before this PR(Seconds) | After this PR(Seconds) -- | -- | -- TPC-DS q58 | 40 | 20 TPC-DS q83 | 18 | 14 Closes #33664 from wangyum/SPARK-36444. Authored-by: Yuming Wang <yumwang@ebay.com> Signed-off-by: Yuming Wang <yumwang@ebay.com> (cherry picked from commit 2310b99) Signed-off-by: Yuming Wang <yumwang@ebay.com>

pwendell added 6 commits March 30, 2014 15:33

Initial pass

9d48cbf

Using better labels

42c1f09

More visiblity clean-up

5a76ed6

Another naming change

d86818e

Marking GraphX as alpha

cf23ec6

Small fixes

542a736

rxin reviewed Apr 3, 2014
View reviewed changes

pwendell added 3 commits April 5, 2014 13:54

Don't use floats

2f706f1

Code review feedback

cd7a465

Formatting and coloring of badges

1ed27d2

Style fixes

8452309

spark.annotations -> spark.annotation

042c803

asfgit closed this in 87bd1f9 Apr 9, 2014

rahij pushed a commit to rahij/spark that referenced this pull request Dec 5, 2017

Merge pull request apache#274 from palantir/aash/resync-apache

d4c6384

[NOSQUASH] Resync Apache

Igosuki pushed a commit to Adikteev/spark that referenced this pull request Jul 31, 2018

Unpin HDFS stub universe (apache#274)

bd0866f

bzhaoopenstack pushed a commit to bzhaoopenstack/spark that referenced this pull request Sep 11, 2019

Merge pull request apache#274 from theopenlab/huaweicloud-openrc

0499873

Add huaweicloud logic for export-cloud-openrc

peter-toth mentioned this pull request Jun 21, 2020

[SPARK-29375][SPARK-28940][SPARK-32041][SQL] Whole plan exchange and subquery reuse #28885

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SPARK-1093: Annotate developer and experimental API's #274

SPARK-1093: Annotate developer and experimental API's #274

pwendell commented Mar 31, 2014

AmplabJenkins commented Mar 31, 2014

AmplabJenkins commented Mar 31, 2014

AmplabJenkins commented Mar 31, 2014

AmplabJenkins commented Mar 31, 2014

mateiz commented Apr 3, 2014

mateiz commented Apr 3, 2014

mateiz commented Apr 3, 2014

rxin Apr 3, 2014

AmplabJenkins commented Apr 5, 2014

AmplabJenkins commented Apr 5, 2014

AmplabJenkins commented Apr 5, 2014

AmplabJenkins commented Apr 5, 2014

AmplabJenkins commented Apr 5, 2014

AmplabJenkins commented Apr 5, 2014

AmplabJenkins commented Apr 6, 2014

aarondav commented Apr 9, 2014

mengxr commented Apr 9, 2014

pwendell commented Apr 9, 2014

AmplabJenkins commented Apr 9, 2014

AmplabJenkins commented Apr 9, 2014

AmplabJenkins commented Apr 9, 2014

AmplabJenkins commented Apr 9, 2014

AmplabJenkins commented Apr 9, 2014

AmplabJenkins commented Apr 9, 2014

AmplabJenkins commented Apr 9, 2014

AmplabJenkins commented Apr 9, 2014

pwendell commented Apr 9, 2014

SPARK-1093: Annotate developer and experimental API's #274

SPARK-1093: Annotate developer and experimental API's #274

Conversation

pwendell commented Mar 31, 2014

AmplabJenkins commented Mar 31, 2014

AmplabJenkins commented Mar 31, 2014

AmplabJenkins commented Mar 31, 2014

AmplabJenkins commented Mar 31, 2014

mateiz commented Apr 3, 2014

mateiz commented Apr 3, 2014

mateiz commented Apr 3, 2014

rxin Apr 3, 2014

Choose a reason for hiding this comment

AmplabJenkins commented Apr 5, 2014

AmplabJenkins commented Apr 5, 2014

AmplabJenkins commented Apr 5, 2014

AmplabJenkins commented Apr 5, 2014

AmplabJenkins commented Apr 5, 2014

AmplabJenkins commented Apr 5, 2014

AmplabJenkins commented Apr 6, 2014

aarondav commented Apr 9, 2014

mengxr commented Apr 9, 2014

pwendell commented Apr 9, 2014

AmplabJenkins commented Apr 9, 2014

AmplabJenkins commented Apr 9, 2014

AmplabJenkins commented Apr 9, 2014

AmplabJenkins commented Apr 9, 2014

AmplabJenkins commented Apr 9, 2014

AmplabJenkins commented Apr 9, 2014

AmplabJenkins commented Apr 9, 2014

AmplabJenkins commented Apr 9, 2014

pwendell commented Apr 9, 2014